Rafting Raft Workshop

Last week, May 2-6, 2022, I attended Rafting Raft workshop by David Beazley. The workshop focussed on building a raft implementation and it was intense and exhausting. A few folks had asked me about the workshop and how it works. The post focuses on what happens before and during the workshop, so future attendees can decide. Day -43 Someday when you’re contemplating attending the workshop, you register on the website. You get a confirmation email from David Beazley to confirm your availability to attend the workshop. [Read More]

Profiling Django App

TL:DR Pyinstrument is a call stack sampling profiler with low overhead to find out time spent in your Django application. QueryCount is a simplistic ORM query count middleware that counts the number of ORM queries, finds duplicate queries, and prints them in the terminal. Django Silk is an extensive Django profiler that records the entire execution, SQL queries, source of origin, and persists the recordings. The complete Django profiler. 🔬 What’s Profiling? [Read More]

HTTPie and Print HTTP Request

HTTPie is a command-line utility for making HTTP requests with more straightforward syntax(controversial, I agree). The interesting feature is --offline flag which prints HTTP raw request text. The client sends the HTTP request to the server, and the server responds to the request. It’s an alternate to curl. HTTP Syntax HTTP Flow and syntrax from Wikipedia. A client sends request messages to the server, which consist of a request line, consisting of the case-sensitive request method, a space, the request target, another space, the protocol version, a carriage return, and a line feed (e. [Read More]
Python  HTTP  CLI  HTTPie 

Type Check Your Django Application

Recently, I gave a talk, Type Check your Django app at two conferences - Euro Python 2021 and PyCon India 2021. The talk was about adding Python gradual typing to Django using third-party package Django-stubs focussed heavily around Django Models. The blog post is the write-up of the talk. Here is the unofficial link recorded video of the PyCon India talk. Here is the link to PyCon India Slides. The slides to Euro Python Talk (both slides are similar). [Read More]

Pulse Plus

PhonePe recently released Pulse repo from their payment data. It was hard to get an overview of the data without doing some data transformation. The data is eight levels deep, nested, and multiple files for similar purpose data. Hard to do any command-line aggregate queries for data exploration. It’s hard to do any analysis with 2000+ files. So I created an SQLite database of the data using python sqlite-utils. The SQLite database aggregated data and top data in 5 tables - aggregated_user, aggregated_user_device, aggregated_transaction, top_user, top_transaction. [Read More]

TIL - A new site

Quite often, as a programmer, I learn something new. Some are utilitarian; some are philosophical; some are opinions in programming. I want to document these learnings for later use and also to remember. So I’m starting a new site, til.kracekumar.com, to demonstrate this learning. So far, there are six posts. The inspiration comes from Simon Willson’s TIL website. Why the new site? Two reasons I’m planning to write often; sometimes, the post will fit in just two tweets. [Read More]

Python Typing Koans

Python 3 introduced type annotation syntax. PEP 484 introduced a provisional module to provide these standard definitions and tools, along with some conventions for situations where annotations are not available. Python is a dynamic language and follows gradual typing. When a static type checker runs a python code, the type checker considers code without type hints as Any. def print_name(name): print(name) planet: str = "earth" In the above example, the name argument type hint will be Any since the type hint is missing while the type hint for the planet variable is a string. [Read More]

Model Field - Django ORM Working - Part 2

The last post covered the structure of Django Model. This post covers how the model field works, what are the some important methods and functionality and properties of the field. Object-Relational Mapper is a technique of declaring, querying the database tables using Object relationship in the programming language. Here is a sample model declaration in Django. class Question(models.Model): question_text = models.CharField(max_length=200) pub_date = models.DateTimeField('date published') Each class inherits from models.Model becomes a table inside the SQL database unless explicitly marked as abstract. [Read More]

Structure - Django ORM Working - Part 1

Django ORM hides a lot of complexity while developing the web application. The data model declaration and querying pattern are simplified, whereas it’s structured differently behind the scenes. The series of blog posts will explain Django ORM working(not just converting Python code to SQL), model declaration, querying (manager, queryset), supporting multiple drivers, writing custom queries, migrations etc… Consider a model definition from the Django tutorial. from django.db import models class Question(models. [Read More]

jut - render jupyter notebook in the terminal

The Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. Uses include: data cleaning and transformation, numerical simulation, statistical modeling, data visualization, machine learning, and much more. The definition copied from the official website. It’s becoming common to use Jupyter notebook to write books, do data analysis, reproducible experiments, etc… The file produced out of notebook follows JSON Schema. [Read More]