NumPy basics with a lot of examples

Photo by Boitumelo Phetla on Unsplash

Note: Most of the examples used to explain concepts of NumPy have been taken from Python For Data Analysis by Wes McKinney.

Let’s get started.

A ndarray is a generic multidimensional container for homogeneous data; that is, all of the elements must be the same type.

Every array has a shape, a tuple indicating the size of each dimension, and a dtype, an object describing the data type of the array:

# randn returns elements from a standard normal distributiondata = np.random.randn(2,3)
data

Output:

array([[-1.00945873, -0.14747028,  1.04654565],
[-0.69762101, 0.35370184, -0.08946465]])

To check the type of each element we use:


How to scrape data from a website and dump it into a CSV

Photo showing random web data
Photo showing random web data
Photo by Markus Spiske on Unsplash

Web Scraping is the process of gathering information from the internet.

Note: If you are scraping a page that is out there on the World Wide Web just for educational purposes, then it seems fine. But still, you should consider checking their Terms of service as few websites don’t like it when automatic scrapers gather their data, while others don’t mind.

Let me give you an easy example of where it can be used. Say you want to buy a popular product from a website that goes out of stock as soon as it comes up. …


Understanding shallow and deep copy in python

A picture showing random python code
A picture showing random python code
A picture showing random python code

While I was working on python I tried copying a list to a new variable by using the ‘=’ operator. After making a few changes I got to know this is not the right way we do it in python.

A few seconds later I was scrolling through multiple articles understanding 2 terminologies used while copying data in python: shallow and deep copy.

The = operator does not copy the object, it just creates a new variable and shares the reference of the original variable.

Note: All the examples used below have been taken from python-engineer and programiz.

list_a =…

Building your own Linear Regression Models with Scikit-learn and the maths behind it

Just another maths GIF

Let's first start with the most basic and most understood regression: Linear Regression. In simpler terms, linear regression attempts to model the relationship between two variables by fitting a linear equation to observed data.

More generally, a linear regression model predicts by simply computing a weighted sum of the input features + constant called the bias term (also known as the intercept term).

Mathematically:

Linear Regression Model Prediction Equation
Linear Regression Model Prediction Equation
Linear Regression Model Prediction Equation

My First Kaggle notebook submission

So after a month of revising concepts of Python, understanding the basics of a few python data handling & visualization libraries like NumPy, Pandas, Matplotlib, etc. I decided to create my account on Kaggle and start my Data Analytics journey with real data.

After signing up I started looking into some popular datasets for which people have submitted their notebooks. I looked into a few of them(high rated). They have been using seaborn, Plotly, etc. to have better visualizations and easy-to-code Plots.

I took another week and immersed myself in concepts of Seaborn and Plotly. After working on the basics…


How our decisions depend upon what is presented to us

Picture depicting framing effect
Picture depicting framing effect

I’ve seen a lot of times how similar data is shown in different ways to bias its overall effect.

Tricks can be performed to frame a particular set of results, converting its outcome from positive to negative or vice-versa.

The FRAMING EFFECT is when our decisions are influenced by the way information is presented. Equivalent information can be more or less attractive depending on what features are highlighted. Decisions based on this framing effect can put something which is of lesser value or importance into a positive light and highly important information into a negative one.

5% mortality sounds worse…


Matplotlib to the rescue

Data visualization is one of the key skills expected these days while working with data. It helps in simplifying complex data into an understandable format and making decisions based on that.

Companies are collecting various types of data including climate data, user data, transactional data, medical data, etc. All these data are analyzed and visualized further to make some very important business decisions.

Visualization works from a human perspective because we respond to and process visual data better than any other type of data. In fact, the human brain processes image 60,000 times faster than text, and 90 percent of…


Handling missing information while working with data

While doing data analysis, a major chunk of your time (around 80%) is spent on loading, cleaning, transforming, rearranging data, and doing all kinds of stuff to bring it in the ‘right format’ state.

Most commonly it is missing data that needs to be handled.

In pandas, we define missing data as NA (not available). We can also use None value as it also treated the same way.

Image showing Nan None
Image showing Nan None
Image showing Nan None

Let’s try to understand it with an example:

# import all the required libraries on your jupyter notebook
import pandas as pd
import numpy as np
from numpy import nan as NA


Changing Its Theme…

Finally, you have made up your mind to install Jupyter notebooks on your system and started the journey of working with data.

Seeing how much Dark Mode is in trend these days, every second application I look into is trying to build this where you can switch from the boring “All White” theme to a Dark Theme (best results at night) with just a toggle switch.

If you are a programmer just like me who likes to work at night on some personal projects, I must tell you have hit a golden nugget by reaching out to this article.

The…

Rahul Kapoor

Implementation Analyst || Learning Analytics and Machine Learning

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store