19 10 / 2014

Python’s global keyword allows to modify the variable which is out of current scope.

In [13]: bar = 1

In [14]: def foo():
....:     global bar
....:     bar = 2
....:

In [15]: bar
Out[15]: 1

In [16]: foo()

In [17]: bar
Out[17]: 2

In the above example, bar was declared before foo function. global bar refers to the bar variable which is outside the foo scope. After foo invocation bar value was modified inside foo. The value is modified globally.

What happens when bar becomes function inside foo ?

In [19]: bar = None

In [20]: def foo():
   ....:     global bar
   ....:     def bar():
   ....:         return 'I am bar'
   ....:

In [21]: bar

In [22]: foo()

In [23]: bar
Out[23]: <function __main__.bar>

bar was variable outside foo's scope. Now creating a new function bar inside foo aliased bar function to bar global variable.

This may not be obvious at first step, then you realize function can be aliased to variable.

23 5 / 2014

Some of the python source file starts with -*- coding: utf-8 -*-. This particular line tells python interpreter all the content (byte string) is utf-8 encoded. Lets see how it affects the code.

uni1.py:

# -*- coding: utf-8 -*-
print("welcome")
print("animé")

output:

➜  code$ python2 uni1.py
   welcome
   animé

Third line had a accented character and it wasn’t explictly stated as unicode. print function passed successfully. Since first line instructed interpreter all the sequences from here on will follow utf-8, so it worked.

What if first line was missing ?

uni2.py

print("welcome")
print("animé")

output:

code$  python2 uni2.py
File "uni2.py", line 2
SyntaxError: Non-ASCII character '\xc3' in file uni2.py on line 2, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details

Now python complains that Non-ASCII character is found since default encoding is ASCII. More about source encoding can be found in PEP 263

Always set encoding in first or second line of python file.

13 5 / 2014

As of writing (12, May 2014) latest version of pip is 1.5.1. pip doesn’t allow installing packages from non PyPI based url. It is possible to upload tar or zip or tar.gz file to PyPI or specify download url which points other sites(Example: pyPdf points to http://pybrary.net/pyPdf/pyPdf-1.13.tar.gz). pip considers externally hosted packages as insecure. Agreed.

This is one of the reason why I kept using pip 1.4.1. Finally decided to fix this issue. Below is the sample error which pip throws.

(document-converter)➜  document-converter git:(fix_requirements) pip install pyPdf
Downloading/unpacking pyPdf
Could not find any downloads that satisfy the requirement pyPdf
Some externally hosted files were ignored (use --allow-external pyPdf to allow).
Cleaning up...
No distributions at all found for pyPdf
Storing debug log for failure in /Users/kracekumar/.pip/pip.log

(document-converter)➜  document-converter git:(fix_requirements) pip install --allow-external pyPdf
You must give at least one requirement to install (see "pip help install")
(document-converter)➜  document-converter git:(fix_requirements) pip install pyPdf --allow-external pyPdf
Downloading/unpacking pyPdf
Could not find any downloads that satisfy the requirement pyPdf
Some insecure and unverifiable files were ignored (use --allow-unverified pyPdf to allow).
Cleaning up...
No distributions at all found for pyPdf
Storing debug log for failure in /Users/kracekumar/.pip/pip.log

The above method is super confusing and counter intutive. Fix is

(document-converter)➜  document-converter git:(fix_requirements) pip install pyPdf --allow-external pyPdf --allow-unverified pyPdf
Downloading/unpacking pyPdf
pyPdf an externally hosted file and may be unreliable
pyPdf is potentially insecure and unverifiable.
Downloading pyPdf-1.13.tar.gz
Running setup.py (path:/Users/kracekumar/Envs/document-converter/build/pyPdf/setup.py) egg_info for package pyPdf

Installing collected packages: pyPdf
Running setup.py install for pyPdf

Successfully installed pyPdf
Cleaning up...

The above method is not used in production environment. In production environment it is recommended to do pip install -r requirements.txt.

# requirements.txt
--allow-external pyPdf
--allow-unverified pyPdf
pyPdf
--allow-external xhtml2pdf==0.0.5

pyPdf has two issues, two flags needs to mentioned in requirements.txt. Since xhtml2pdf requires pyPdf --allow-external flag is passed. Wish it was possible to pass both the switches in same line. If you do so pip will ignore it. Now running pip install -r requirements.txt will works like a charm(with warnings).

Since current approach is super confusing, there is a discussion. Thanks Ivoz for helping me to resolve this.

07 4 / 2014

Over period of time few people have asked me in meetups, online I want to learn python. Suggest me few ways to learn. Everyone who asked me had different background and different intentions. Before answering the question I try to collect more information about their interest and their previous approaches. Some learnt basics from codecademy, some attended beginners session in Bangpypers meetup. In this post I will cover general questions asked and my suggested approach.

Q: Suggest some online resources and books to learn python ?

A: I suggest three resources, How to think like computer scientist, Learn Python The Hardway, CS101 from Udacity. This is highly subjective because it depends on previous programming experience etc … I have a blog post with lot of python snippet without explanation (I know it is like sea without waves).

Q: It takes too much time to complete the book. I want to learn it soon.

A: I have been programming in python over 3 years now, still I don’t know in depth python. You may learn it in six months or in a week. It is the journey which is interesting more than destination.

Q: How long will it take to learn python ?

A: It depends what you want to learn in python. I learnt python in 3 hours while commuting to college. You should be able to grasp basic concepts in few hours. Practice will make you feel confident.

Q: I learnt basics of Python, can you give me some problems to solve and I will get back with solutions ?

A: No. I would be glad to help you if you are stuck. Learning to solve your problem is great way to learn. My first usable python program was to download Tamil songs. I still use the code :-). So find the small problem or project to work on. I would be happy to review the code and give suggestions to it.

Q: I want to contribute to open source python projects can you suggest ?

A: Don’t contribute to project because you want to, rather find a library or project which is interesting to you and see if things can be made better. Your contribution can be small like fixing spelling mistake (I have contributed with single character change). Linux kernel accepts patch which fixes spelling mistake. Every contribution has its own effect. So contribute if it adds value.

In case you are reading this blog post and want to learn python or need help I would be glad to help.

22 3 / 2014

We are writing a small utility function called is_valid_mime_type. The function takes a mime_type as an argument and checks if the mime type is one of the allowed types. Code looks like

ALLOWED_MIME_TYPE = ('application/json', 'text/plain', 'text/html')

def is_valid_mimetype(mime_type):
    """Returns True or False.

    :param mime_type string or unicode: HTTP header mime type
    """
    for item in ALLOWED_MIME_TYPE:
        if mime_type.startswith(item):
            return True
    return False

Above code can refactored into single line using any.

def is_valid_mimetype(mime_type):
    """Returns True or False.

    :param mime_type string or unicode: HTTP header mime type
    """
    return any([True for item in ALLOWED_MIME_TYPE
                              if mime_type.startswith(item)])

One liner. It is awesome, but not performant. How about using next ?

def is_valid_mimetype(mime_type):
    """Returns True or False.

    :param mime_type string or unicode: HTTP header mime type
    """
    return next((True for item in ALLOWED_MIME_TYPE 
                              if mime_type.startswith(item)), False)

(True for item in ALLOWED_MIME_TYPE if mime_type.startswith(item) is generator expression. When ALLOWED_MIME_TYPE is None or EMPTY exception will be raised. In order to avoid that False is passed as an argument to next.

Edit:

def is_valid_mimetype(mime_type):
    """Returns True or False.

    :param mime_type string or unicode: HTTP header mime type
    """
    return any(mime_type.startswith(item) for item in ALLOWED_MIME_TYPE)

Cleaner than Next.

07 3 / 2014

Python has sorted function which sorts iterable in ascending or descending order.

# Sort descending
In [95]: sorted([1, 2, 3, 4], reverse=True)
Out[95]: [4, 3, 2, 1]

# Sort ascending
In [96]: sorted([1, 2, 3, 4], reverse=False)
Out[96]: [1, 2, 3, 4]

sorted(iterable, reverse=True)[:n] will yield first n largest numbers. There is an alternate way.

Python has heapq which implements heap datastructure. heapq has function nlargest and nsmallest which take arguments n number of elements, iterable like list, dict, tuple, generator and optional argument key.

In [85]: heapq.nlargest(10, [1, 2, 3, 4,])
Out[85]: [4, 3, 2, 1]

In [88]: heapq.nlargest(10, xrange(1000))
Out[88]: [999, 998, 997, 996, 995, 994, 993, 992, 991, 990]

In [89]: heapq.nlargest(10, [1000]*10)
Out[89]: [1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000]

In [99]: heapq.nsmallest(3, [-10, -10.0, 20.34, 0.34, 1])
Out[99]: [-10, -10.0, 0.34]

Let’s say marks is a list of dictionary containing students marks. Now with heapq it is possible to find highest and lowest mark in a subject.

In [113]: marks = [{'name': "Ram", 'chemistry': 23},{'name': 'Kumar', 'chemistry': 50}, {'name': 'Franklin', 'chemistry': 89}]

In [114]: heapq.nlargest(1, marks, key=lambda mark: mark['chemistry'])
Out[114]: [{'chemistry': 89, 'name': 'Franklin'}]

In [115]: heapq.nsmallest(1, marks, key=lambda mark: mark['chemistry'])
Out[115]: [{'chemistry': 23, 'name': 'Ram'}]

heapq can be used for building priority queue.

Note: IPython is used in examples where In [114]: means Input line number 114 and Out[114] means Output line number 114.

27 2 / 2014

Let’s say you want to find how many times each element is present in the list or tuple.

Normal approach

words = ['a', 'the', 'an', 'a', 'an', 'the']
d = {}
for word in words:
    if word in d:
        d[word] += 1
    else:
        d[word] = 1
print d
{'a': 2, 'the': 2, 'an': 2} 

Better approach

words = ['a', 'the', 'an', 'a', 'an', 'the']
d = {}
for word in words:
    d[word] = d.get(word, 0) + 1

print d
{'a': 2, 'the': 2, 'an': 2

Both the approach returned same values. The first one has 6 lines of logic and second has 3 lines of logic (less code less management).

Second approach uses d.get method. d.get(word, 0) return count of the word if key is present else 0. If 0 isn’t passed get will return None.

Pythonic approach:

import collections

words = ['a', 'b', 'a']

res = collections.Counter(words)

print res
Counter({'a': 2, 'b': 1})

Last approach is just one line and pythonic.

Snippet is extracted from Transforming Code into Beautiful, Idiomatic Python. Do watch and enjoy.

08 2 / 2014

It is very common to update single attribute of a model instance (say update first name in user profile) and save it to db.

In [18]: u = User.objects.get(id=1)

In [19]: u.first_name = u"kracekumar"

In [20]: u.save()

Very straight forward approach. How does django send the sql query to database ?

In [22]: from django.db import connection

In [22]: connection.queries
Out[22]: 
[... 
{u'sql': u'UPDATE "auth_user" SET "password" = \'pbkdf2_sha256$12000$vsHWOlo1ZhZg$DrC46wq+a2jEtEzxmUEw4vQw8oV/rxEK7zVi30QLGF4=\', "last_login" = \'2014-02-01 06:55:44.741284+00:00\', "is_superuser" = true, "username" = \'kracekumar\', "first_name" = \'kracekumar\', "last_name" = \'\', "email" = \'me@kracekumar.com\', "is_staff" = true, "is_active" = true, "date_joined" = \'2014-01-30 18:41:18.174353+00:00\' WHERE "auth_user"."id" = 1 ', u'time': u'0.001'}]

Not happy. Honestly it should be UPDATE auth_user SET first_name = 'kracekumar' WHERE id = 1. Django should ideally update modified fields.

Right way to do is

In [23]: User.objects.filter(id=u.id).update(first_name="kracekumar")
Out[23]: 1

In [24]: connection.queries
Out[24]:
[...
{u'sql': u'UPDATE "auth_user" SET "first_name" = \'kracekumar\' WHERE "auth_user"."id" = 1 ', u'time': u'0.001'}]

Yay! Though both queries took same amount of time, latter is better.

Edit: There is one more cleaner way to do it.

In [60]: u.save(update_fields=['first_name'])

In [61]: connection.queries
Out[61]: 
[...
{u'sql': u'UPDATE "auth_user" SET "first_name" = \'kracekumar\'  WHERE "auth_user"."id" = 1 ',
u'time': u'0.001'}]

26 12 / 2013

This is the material which I use for teaching python to beginners.

tld;dr: Very minimal explanation more code.

Python?

  • Interpreted language
  • Multiparadigm

Introduction

hasgeek@hasgeek-MacBook:~/codes/python/hacknight$ python
Python 2.7.3 (default, Aug  1 2012, 05:14:39) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>


>>> print "Let's learn Python"
Let's learn Python

Numbers

>>> 23 + 43
66
>>> 23 - 45
-22
>>> 23 * 45
1035
>>> 23 ** 4
279841
>>> 23 / 4
5
>>> 23 / 4.0
5.75
>>> 7 % 2
1

Expressions

>>> 3 < 2
False
>>> 3 > 2
True
>>> 3 > 2 < 1
False
>>> (3 > 2) and (2 < 1)
False
>>> 3 > 2 > 1 > 0
True
>>> (3 > 2) and (2 > 1) and (1 > 0)
True
>>> 1 or 2
1
>>> 2 or 1
2
>>> 1 + 2 + 3 * 4 + 5
20
1 + 2 + 3 * 4 + 5
↓
3   + 3 * 4 + 5
        ↓
3   +   12  + 5
      ↓
    15      + 5
          ↓
         20

>>> "python" > "perl"
True
>>> "python" > "java"
True

Variables

>>> a = 23
>>> print a
23
>>> a = "Python"
>>> print a
Python

Guess the output

True = False
False = True
print True, False
print 2 > 3

Parallel Assignment

>>> language, version = "Python", 2.7
>>> print language, version
Python 2.7
>>> x = 23
>>> x = 23
>>> y = 20
>>> x, y = x, x + y
>>> print x, y
23 43

Guess the output

z, y = 23, z + 23
a, b = 23, 12, 20
a = 1, 2

Swap Variable

>>> x = 12
>>> y = 21
>>> x, y = y, x
>>> print x, y
21 12
>>>

String

>>> language = "Python"
>>> print language
Python
>>> language = 'Python'
>>> print language
Python
>>> language = """Python"""
>>> print language
Python
>>> description = """Python is a general-purpose, high-level programming language whose design philosophy emphasizes code readability.
... It is an expressive language which provides language constructs intended to enable clear programs on both a small and large scale.
... Python supports multiple programming paradigms, including object-oriented, imperative and functional programming styles.
... """
>>> print description
Python is a general-purpose, high-level programming language whose design philosophy emphasizes code readability.
It is an expressive language which provides language constructs intended to enable clear programs on both a small and large scale.
Python supports multiple programming paradigms, including object-oriented, imperative and functional programming styles.
>>> 

Guess output

name = "krace" + "kumar"
print name
print name[0]
name[0] = "K"

Guess output

print 1 + 2.5
print "kracekumar" + 23

Condition

Write a program to find greatest of two numbers.

>>> a = 12
>>> b = 23
>>> if a > b:
...     print "a is greater than b"
... else:
...     print "b is greater than a"
... 
b is greater than a
>>> if a > 0:
...     print "a is positive"
... elif a == 0:
...     print "a is zero"
... elif a < 0:
...     print "a is negative"
... 
a is positive

Data Structure

List

List is a collection of heterogenous data types like integer, float, string.

>>> a = [1, 2, 3]
>>> b = ["Python", 2.73, 3]
>>> len(a)
3
>>> len(b)
3
>>> a[0]
1
>>> a[-1]
3
>>> b[2]
3
>>> [1, 2] + [3, 4]
[1, 2, 3, 4]
>>> all = [a, b]
>>> all[0]
[1, 2, 3]
>>> all[-1]
['Python', 2.73, 3]
>>> all[3]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IndexError: list index out of range
>>> all.append("Bangalore")
>>> all
[[1, 2, 3], ['Python', 2.73, 3], 'Bangalore']
>>> del all[-1]
>>> all
[[1, 2, 3], ['Python', 2.73, 3]]
>>> all[1] = "insert"
>>> all
[[1, 2, 3], 'insert']
>>> all
[[1, 2, 3], 'insert']
>>> 'insert' in all
True
>>> range(10)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> range(10, 2)
[]
>>> range(10, 0, -1)
[10, 9, 8, 7, 6, 5, 4, 3, 2, 1]
>>> range(0, 12, 1)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]



range() -> `range([start,] stop[, step]) -> list of integers`

Slicing

>>> l = [1, 2, 3, 4, 5, 6, 7]
[1, 2, 3, 4, 5, 6, 7]
>>> l[:2] #first two elements
[1, 2]
>>> l[2:] #exclude first two elements
[3, 4, 5, 6, 7]
>>> l[::2] #every second element
[1, 3, 5, 7]
>>> l[::1] #every element
[1, 2, 3, 4, 5, 6, 7]
>>> l[::3] #every third element
[1, 4, 7]
>>> l[::10] #every tenth element
[1]
>>> l[::-1]
[7, 6, 5, 4, 3, 2, 1]

Guess the output

l[1:7:2] [][:2] [1][:2]

Accessing list elements

>>> for item in all:
...     print item
... 
[1, 2, 3]
insert
>>> for number in range(10):
...     print number
... 
0
1
2
3
4
5
6
7
8
9

Find all odd numbers from 0 to 9

>>> for number in range(0, 10):
...     if number % 2:
...         print number
... 
1
3
5
7
9

inbuilt functions

>>> help([])
>>> min([1, 2, 3])
1
>>> max([1, 2, 3])
3
>>> sum([1, 2, 3])
6
>>> pow(2, 3)
8

Write program which takes a number as input and if number is divisible by 3, 5 print Fizz, Buzz, FizzBuzz respectively

import sys


if __name__ == "__main__":
    if len(sys.argv) == 2:
        number = int(sys.argv[1])
    if number % 15 == 0:
        print "FizzBuzz"
    elif number % 3 == 0:
        print "Fizz"
    elif number % 5 == 0:
        print "Buzz"
    else:
        print number

else:
    print "python filename.py 23 is the format"

Tuples

Tuple is a sequence type just like list, but it is immutable. A tuple consists of a number of values separated by commas.

>>> t = (1, 2)
>>> t
(1, 2)
>>> t[0]
1
>>> t[0] = 1.1
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment
>>> t = 1, 2
>>> t
(1, 2)
>>> del t[0]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'tuple' object doesn't support item deletion
>>> for item in t:
...     print item
... 
1
2

Sets

Sets are unordered collection of unique elements.

>>> x = set([1, 2, 1])
>>> x
set([1, 2])
>>> x.add(3)
>>> x
set([1, 2, 3])
>>> x = {1, 3, 4, 1}
>>> x
set([1, 3, 4])
>>> 1 in x
True
>>> -1 in x
False
>>> 

Again Lists

>>> even_numbers = []
>>> for number in range(0, 9):
...     if number % 2 == 0:
...         even_numbers.append(number)
... 
>>> even_numbers
[0, 2, 4, 6, 8]

As a programmer your job is write lesser code

List Comprehensions

>>> [x for x in range(10)]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> [x + 1 for x in range(10)]
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
>>> numbers = []
>>> for x in range(10):
...     numbers.append(x + 1)
... 
>>> print numbers
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
>>> even_numbers = [x for x in range(10) if x %2 == 0]
>>> even_numbers
[0, 2, 4, 6, 8]
>>> [(x, y) for x in range(5) for y in range(5) if (x+y)%2 == 0]
[(0, 0), (0, 2), (0, 4), (1, 1), (1, 3), (2, 0), (2, 2), (2, 4), (3, 1), (3, 3    ), (4, 0), (4, 2), (4, 4)]
>>> 

Dictionaries

>>> d = {'a': 1, 'b': 2, 'c': 3}
>>> d['a']
1
>>> d.get('a')
1
>>> d['z']
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
KeyError: 'z'
>>> d.get('z')
>>>
>>> d['a'] = 2
>>> d
{'a': 2, 'c': 3, 'b': 2}
>>> d['z'] = 26
>>> d
{'a': 2, 'c': 3, 'b': 2, 'z': 26}
>>> d.keys()
['a', 'c', 'b', 'z']
>>> d.values()
[2, 3, 2, 26]
>>> d.items()
[('a', 2), ('c', 3), ('b', 2), ('z', 26)]
>>> type(d.items())
<type 'list'>
>>> d = {'a': 2, 'b': 2, 'c': 3, 'z': 26}
>>> for key in d:
...     print key
... 
a
c
b
z
>>> for key, value in d.items():
...     print key, value
... 
a 2
c 3
b 2
z 26
>>> 'a' in d
True
>>> d.has_key('a')
True

Function

Just like a value can be associated with a name, a piece of logic can also be associated with a name by defining a function.

>>> def square(x):
...     return x * x
... 
>>> square(2)
4
>>> square(2+1)
9
>>> square(x=5)
25
>>> def dont_return(name):
...     print "Master %s ordered not to return value" % name
... 
>>> dont_return("Python")
Master Python ordered not to return value
>>> def power(base, to_raise=2):
...     return base ** to_raise
... 
>>> power(3)
9
>>> power(3, 3)
27
>>> def power(to_raise=2, base):
...     return base ** to_raise
... 
File "<stdin>", line 1
SyntaxError: non-default argument follows default argument 
>>> square(3) + square(4)
25
>>> power(base=square(2))
16
>>> def sum_of_square(x, y):
...     return square(x) + square(y)
... 
>>> sum_of_square(2, 3)
13
>>> s = square
>>> s(4)
16
>>> def fxy(f, x, y):
...     return f(x) + f(y)
... 
>>> fxy(square, 3, 4)
25

Methods - Methods are special kind of functions that work on an object.

>>> lang = "Python"
>>> type(lang)
<type 'str'>
>>> dir(lang)
['__add__', '__class__', '__contains__', '__delattr__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__',
'__getitem__', '__getnewargs__', '__getslice__', '__gt__', '__hash__', '__init__', '__le__', '__len__', '__lt__',
'__mod__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__rmod__', '__rmul__', '__setattr__',
'__sizeof__', '__str__', '__subclasshook__', '_formatter_field_name_split', '_formatter_parser',
'capitalize', 'center', 'count', 'decode', 'encode', 'endswith', 'expandtabs', 'find', 'format', 'index', 'isalnum',
'isalpha', 'isdigit', 'islower', 'isspace', 'istitle', 'isupper', 'join', 'ljust', 'lower', 'lstrip', 'partition',
'replace', 'rfind', 'rindex', 'rjust', 'rpartition', 'rsplit', 'rstrip',     'split', 'splitlines',
'startswith', 'strip', 'swapcase', 'title', 'translate', 'upper', 'zfill']
>>> lang.upper()
'PYTHON'
>>> help(lang.upper)
>>> lang.startswith('P')
True
>>> help(lang.startswith)

>>> lang.startswith('y', 1)
True

Files

>>> f = open('foo.txt', 'w')
>>> help(f)

>>> f.write("First line")
>>> f.close()
>>> f = open('foo.txt', 'r')
>>> f.readline()
'First line'
>>> f.readline()
''
>>> f = open('foo.txt', 'a')
>>> f.write('Second line')
>>> f.close()
>>> f = open('foo.txt', 'r')
>>> f.readline()
'First lineSecond line'
>>> f = open('foo.txt', 'a')
>>> f.write("New line\n")
>>> f.write("One more new line")
>>> f.close()
>>> f = open('foo.txt', 'r')
>>> f.readline()
'First lineSecond lineNew line\n'
>>> f.readline()
'One more new line'
>>> f.readline()
''
>>> f.close()
>>> f = open('foo.txt')
>>> f.readlines()
['First lineSecond lineNew line\n', 'One more new line']
>>> f = open('foo.txt', 'w')
>>> f.writelines(["1\n", "2\n"])
>>> f.close()
>>> f.readlines()
>>> f = open('foo.txt')
>>> f.readlines()
['1\n', '2\n']
>>> f.close()

Exception Handling

>>> f = open('a.txt')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IOError: [Errno 2] No such file or directory: 'a.txt'
>>> try:
...     f = open('a.txt')
... except:
...     print "Exception occured"
... 
Exception occured
>>> try:
...     f = open('a.txt')
... except IOError, e:
...     print e.message
... 

>>> e
IOError(2, 'No such file or directory')
>>> dir(e)
['__class__', '__delattr__', '__dict__', '__doc__', '__format__', '__getattribute__', '__getitem__', '__getslice__', '__hash__', '__init__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setstate__', '__sizeof__', '__str__', '__subclasshook__', '__unicode__', 'args', 'errno',     'filename', 'message', 'strerror']
>>> e.strerror
'No such file or directory'
>>> try:
...     print l[4]
... except IndexError, e:
...     print e
... 
list index out of range
>>> raise Exception("error message")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
Exception: error message
>>> try:
...     print "a"
...     raise Exception("doom")
... except:
...     print "b"
... else:
...     print "c"
... finally:
...     print "d"
... 
a
b
d

Object Oriented Programming

>>> class BankAccount:
        def __init__(self):
            self.balance = 0

        def withdraw(self, amount):
            self.balance -= amount
            return self.balance

        def deposit(self, amount):
            self.balance += amount
            return self.balance

>>> a = BankAccount()
>>> b = BankAccount()
>>> a.deposit(200)
200
>>> b.deposit(500)
500
>>> a.withdraw(20)
180
>>> b.withdraw(1000)
-500
>>> class MinimumBalanceAccount(BankAccount):
...    def __init__(self, minimum_balance):
...        BankAccount.__init__(self)
...        self.minimum_balance = minimum_balance
...
...    def withdraw(self, amount):
...        if self.balance - amount < self.minimum_balance:
...            print "Sorry, you need to maintain minimum balance"
...        else:
...            return BankAccount.withdraw(self, amount)
>>> a = MinimumBalanceAccount(500)
>>> a
<__main__.MinimumBalanceAccount instance at 0x7fa0bf329878>
>>> a.deposit(2000)
2000
>>> a.withdraw(1000)
1000
>>> a.withdraw(1000)
Sorry, you need to maintain minimum balance 
>>> class A:
...     def f(self):
...         return self.g()
...     def g(self):
...         return "A"
... 
>>> a = A()
>>> a.f()
'A'
>>> a.g()
'A'
>>> class A:
...     def __init__(self):
...         self._protected = 1
...         self.__private = 2
... 
>>> a = A()
>>> a._protected
1
>>> a.__private
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: A instance has no attribute '__private'

Sample Python Program

#! /usr/bin/env python
# -*- coding: utf-8 -*-


class BankAccount:
    def __init__(self):
        self.balance = 0

    def withdraw(self, amount):
        self.balance -= amount
        return self.balance

    def deposit(self, amount):
        self.balance += amount
        return self.balance


class MinimumBalanceAccount(BankAccount):
    def __init__(self, minimum_balance):
        BankAccount.__init__(self)
        self.minimum_balance = minimum_balance

    def withdraw(self, amount):
        if self.balance - amount < self.minimum_balance:
            print "Sorry, you need to maintain minimum balance"
        else:
            return BankAccount.withdraw(self, amount)

    def __repr__(self):
        return "MinimuBalanceAccount, Balance: %d" %(self.balance)


if __name__ == "__main__":
    a = MinimumBalanceAccount(500)
    print a.deposit(5000)
    print a.withdraw(4500)
    print a.withdraw(500)

Few examples are taken from python practice book.

Github repo: https://github.com/kracekumar/python-training

25 12 / 2013

This article will focus on deploying flask app starting from scratch like creating separate linux user, installating database, web server. Web server will be nginx, database will be postgres, python 2.7 middleware will be uwsgi, server ubuntu 13.10 x64. Flask app name is fido. Demo is carried out in Digital ocean.

Step 1 - Installation


Python header

root@fido:~# apt-get install -y build-essential python-dev

Install uwsgi dependencies

root@fido:~# apt-get install -y libxml2-dev libxslt1-dev

Nginx, uwsgi

root@fido:~#  apt-get install -y nginx uwsgi uwsgi-plugin-python

Start nginx

root@fido:~# service nginx start
* Starting nginx nginx                                                          [ OK ]

Postgres

root@fido:~# apt-get install -y postgresql postgresql-contrib libpq-dev

Step 2 - User

Create a new linux user fido

root@fido:~# adduser fido

Enter all the required details.

root@fido:~# ls /home
fido

Successfully new user is created.

Grant fido root privilege.

root@fido:~# /usr/sbin/visudo
# User privilege specification

root    ALL=(ALL:ALL) ALL 
fido    ALL=(ALL:ALL) ALL

Since fido is not normal user, delete fido’s home directory.

root@fido:~# rm -rf /home/fido
root@fido:~# ls /home
root@fido:~#

Create a new db user fido

root@fido:~# su - postgres
postgres@fido:~$ createuser --pwprompt
Enter name of role to add: fido
Enter password for new role:
Enter it again:
Shall the new role be a superuser? (y/n) y

--pwprompt will prompt for password. release is the password I typed (we need this to connect db from app).

Create a new database fido

postgres@fido:~$ createdb fido;
postgres@fido:~$ psql -U fido -h localhost
Password for user fido:
psql (9.1.10)
SSL connection (cipher: DHE-RSA-AES256-SHA, bits: 256)
Type "help" for help.
fido=# \d
No relations found.

Done. New database role fido and database is created. We are successfully able to login.

Step 3 - Python dependencies

Install pip

root@fido:# cd /tmp
root@fido:/tmp# wget https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py
root@fido:/tmp# python ez_setup.py install
root@fido:/tmp# easy_install pip 
# What is easy_install ? Python package manager.
# what is pip ? Python package manager.
# How to install pip ? easy_install pip. 
# Shame on python :-(
...
Installed /usr/local/lib/python2.7/dist-packages/pip-1.4.1-py2.7.egg
Processing dependencies for pip
Finished processing dependencies for pip

Install virtualenv

root@fido:/tmp# pip install virtualenv

Step 4 - Install app dependencies

Here is the sample app code. The app is just for demo. The app will be placed in /var/www/fido. Normally in production, this will be similar to git clone <url> or hg clone <url> inside directory. Make sure you aren’t sudo while cloning.

root@fido:/tmp# cd /var
root@fido:/var# mkdir www
root@fido:/var# mkdir www/fido

Change the owner of the repo to fido.

root@fido:/var# chown fido:fido www/fido
root@fido:/var# ls -la www/
total 12
drwxr-xr-x  3 root root 4096 Dec 25 03:18 .
drwxr-xr-x 14 root root 4096 Dec 25 03:18 ..
drwxr-xr-x  2 fido fido 4096 Dec 25 03:18 fido

app.py - fido application.

# /usr/bin/env python

from flask import Flask, request
from flask.ext.sqlalchemy import SQLAlchemy

app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = "postgres://fido:release@localhost:5432/fido"
db = SQLAlchemy(app)


class Todo(db.Model):
    id = db.Column(db.Integer(), nullable=False, primary_key=True)
    name = db.Column(db.UnicodeText(), nullable=False)
    status = db.Column(db.Boolean(), default=False, nullable=True)


@app.route("/")
def index():
    return "Index page. Use /new create a new todo"


@app.route('/new', methods=['POST'])
def new():
    form = request.form
    name, status = form.get('name'), form.get('status') or False
    todo = Todo(name=name, status=status)
    db.session.add(todo)
    db.session.commit()
    return "Created todo: {}".format(name)


if __name__ == "__main__":
    db.create_all()
    app.run('0.0.0.0', port=3333, debug=True)

Add a wsgi file website.py

root@fido:/var/www/fido# cat website.py
import sys
import os.path
sys.path.insert(0, os.path.dirname(__file__))
from app import app as application

Files in fido directory.

root@fido:/var/www/fido# tree .
.
├── app.py
├── __init__.py
└── website.wsgi

0 directories, 3 files

Step 5 - Virtual env and dependencies

root@fido:/var/www/fido# virtualenv --no-site-packages env
root@fido:/var/www/fido# . env/bin/activate
(env)root@fido:/var/www/fido# pip install flask sqlalchemy flask-sqlalchemy psycopg2

Step 6 - final setup

Create uwsgi config file

# Add following lines to fido.ini file
root@fido:/etc# cat uwsgi/apps-enabled/fido.ini
[uwsgi]
socket = 127.0.0.1:5000
threads = 2
master = true
uid = fido
gid = fido
chdir = /var/www/fido
home = /var/www/fido/env/
pp = ..
module = website

Check whether uwsgi is booting up properly.

root@fido:/var/www/fido# uwsgi --ini /etc/uwsgi/apps-enabled/fido.ini
...
...
Python version: 2.7.5+ (default, Sep 19 2013, 13:52:09)  [GCC 4.8.1]
Set PythonHome to /var/www/fido/env/
Python main interpreter initialized at 0xb96a70
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 165920 bytes (162 KB) for 2 cores
*** Operational MODE: threaded ***
added ../ to pythonpath.
WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0xb96a70 pid:  17559 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 17559)
spawned uWSGI worker 1 (pid: 17562, cores: 2)

uwsgi is able to load the app without any issues. Kill the uwsgi using keyboard interrupt.

Now lets create table.

root@fido:/var/www/fido# . env/bin/activate
(env)root@fido:/var/www/fido# python app.py
* Running on http://0.0.0.0:3333/
* Restarting with reloader

Exit the program, db.create_all() must have created the table. Normally in production environment, it is advised to use python manage.py db create or any similar approach.

Configure nginx

root@fido:/var/www/fido# cat /etc/nginx/sites-enabled/fido.in
upstream flask {
    server 127.0.0.1:5000;
}

# configuration of the server
server {
    # the domain name it will serve for
    listen 127.0.0.1; # This is very important to test the server locally
    server_name fido.in; # substitute your machine's IP address or FQDN
    charset     utf-8;

    location / {
        uwsgi_pass  flask;
        include uwsgi_params;
    }
}

Now nginx and uwsgi will be running in background. Restart them.

root@fido:/var/www/fido# service nginx restart
* Restarting nginx nginx                                                             [ OK ]
root@fido:/var/www/fido# service uwsgi restart
* Restarting app server(s) uwsgi                                                    [ OK ]

Machine name is fido, so lets try curl http://fido

root@fido:/var/www/fido# curl http://fido
Index page. Use /new create a new todo

Create a new task.

root@fido:/var/www/fido#curl --data "name=write blog post about flask deployment"  http://fido/new
Created todo: write blog post about flask deployment

We have successfully deployed flask + uwsgi + nginx.

Since we installed uwsgi from ubuntu repo, it is started as upstart process, that is why we issue commands like service uwsgi restart.

To see all upstart service, try service --status-all.

If you are running multiple web application in a single server, create one user per application.