CLI

HTTPie is a command-line utility for making HTTP requests with more straightforward syntax(controversial, I agree). The interesting feature is --offline flag which prints HTTP raw request text. The client sends the HTTP request to the server, and the server responds to the request. It’s an alternate to curl. HTTP Syntax HTTP Flow and syntrax from Wikipedia. A client sends request messages to the server, which consist of a request line, consisting of the case-sensitive request method, a space, the request target, another space, the protocol version, a carriage return, and a line feed (e. [Read More]

PhonePe recently released Pulse repo from their payment data. It was hard to get an overview of the data without doing some data transformation. The data is eight levels deep, nested, and multiple files for similar purpose data. Hard to do any command-line aggregate queries for data exploration. It’s hard to do any analysis with 2000+ files. So I created an SQLite database of the data using python sqlite-utils. The SQLite database aggregated data and top data in 5 tables - aggregated_user, aggregated_user_device, aggregated_transaction, top_user, top_transaction. [Read More]

The Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. Uses include: data cleaning and transformation, numerical simulation, statistical modeling, data visualization, machine learning, and much more. The definition copied from the official website. It’s becoming common to use Jupyter notebook to write books, do data analysis, reproducible experiments, etc… The file produced out of notebook follows JSON Schema. [Read More]