pandas_slides

# Python Workshop:
# Handling data with Pandas
 
## <a href="http://www.tiesdekok.com" target="_blank">Ties de Kok</a>
## Tilburg University

---

---

<img style="position: relative; top: 30px; left: 0px;" src="images/pandas_intro.PNG", width=100%>
 
]

---

<img style="position: relative; top:0px; left: -160px;" src="images/dataframe.PNG", width=120%>
 
]

--
.right-column-next[

]

---

### What are we going to do this session

1. Terminology

2. Specific topics:
  - Open files
  - Saving files
  - Navigating dataframe
  - Select data
  - Create new columns
  - Merge data  
  - Groupby operation
  - Plotting with Pandas
  - Plotting with Seaborn 
    
]

---
class: tocslide

### Pandas vs. Numpy

Numpy provides a powerful N-dimensional array object.

Pandas builds upon the Numpy functionality.

]

--
.right-column-next[

### pd.DataFrame vs. pd.Series

A Pandas Series is a 1D data structure (like a vector)  
A Pandas DataFrame is a 2D data structure (like a matrix)

Columns and rows in a DataFrame are Series.

]

---
class: tocslide

.left-column[
 ## Open data
]
.right-column[
## Opening data
Pandas can open pretty much any data file!

<a href="https://nbviewer.jupyter.org/github/TiesdeKok/LearnPythonforResearch/blob/master/1_opening_files.ipynb">Opening and Saving files with Pandas notebook</a>

]

---
class: tocslide

.left-column[
 ## Open data
 ## Save data
]
.right-column[
## Saving data
Pandas can save to pretty much any data file! (except SAS)

]

---
class: tocslide

Tip: HDF files are awesome!

]

---
class: tocslide

.left-column[
 ## Open data
 ## Save data
 ## Navigate
]
.right-column[
## How to inspect your data?

There is no standard data browser for DataFrames

## My recommendation

Use basic operations to view parts of the data in the notebook:

]

---
class: tocslide

.left-column[
  ## Open data
  ## Save data
  ## Navigate
]
.right-column[
## Alternative, use the QGrid extension

]

---
class: tocslide

.left-column[
 ## Open data
 ## Save data
 ## Navigate
 ## Select data
]
.right-column[
## Selecting data

<a href="https://nbviewer.jupyter.org/github/TiesdeKok/LearnPythonforResearch/blob/master/2_handling_data.ipynb#Select-based-on-condition
">Selecting data based on a condition, Jupyter Notebook</a>

]

---
class: tocslide

.left-column[
 ## Open data
 ## Save data
 ## Navigate
 ## Select data
 ## Create Columns
]
.right-column[
## Creating columns

<img style="position: relative; top:-30px; left: 00px;" src="images/create_cols.PNG", width=100%>
<a href="https://nbviewer.jupyter.org/github/TiesdeKok/LearnPythonforResearch/blob/master/2_handling_data.ipynb#Generate-a-new-column-by-iterating-over-the-dataframe-per-row
">Various methods to create columns, Jupyter Notebook</a>

]

---
class: tocslide

.left-column[
 ## Open data
 ## Save data
 ## Navigate
 ## Select data
 ## Create Columns
 ## Merge data
]
.right-column[
## Merging DataFrames

<img style="position: relative; top:-20px; left: 00px;" src="images/pandas_merge.PNG", width=90%> 
<a href="https://nbviewer.jupyter.org/github/TiesdeKok/LearnPythonforResearch/blob/master/2_handling_data.ipynb#1)-Merge-datasets
">Various methods to merge, join, and append, Jupyter Notebook</a>
]

---
class: tocslide

.left-column[
 ## Open data
 ## Save data
 ## Navigate
 ## Select data
 ## Create Columns
 ## Merge data
 ## GroupBy Operation
]
.right-column[
## GroupBy Operations

<img style="position: relative; top:-20px; left: 00px;" src="images/groupby.PNG", width=75%> 
<a href="https://nbviewer.jupyter.org/github/TiesdeKok/LearnPythonforResearch/blob/master/2_handling_data.ipynb#Group-by-operations-(to-top)
">Various methods to merge, join, and append, Jupyter Notebook</a>

]

---
class: tocslide

.left-column[
 ## Save data
 ## Navigate
 ## Select data
 ## Create Columns
 ## Merge data
 ## Groupby Operation
 ## Plotting
]
.right-column[
## Plotting data (Pandas and Seaborn)

<img style="position: relative; top:-10px; left: 50px;" src="images/plotting.PNG", width=75%> 
<a href="https://nbviewer.jupyter.org/github/TiesdeKok/LearnPythonforResearch/blob/master/3_visualizing_data.ipynb
">Comprehensive notebook for plotting with Pandas</a>

]

---
class: tocslide

Questions?

<img style="position: relative; top: 10px; left:80px;" src="images/Business4-150ppp.jpg", width=60%> 
 
]

---
class: tocslide

Demonstration

]

---
class: tocslide

## Mini Task

**Goal:** Get hands-on experience with Pandas on a real-life dataset.

### Instructions

1. Open (start) a Jupyter Notebook in the `UW_python_2018` folder  
2. Solve tasks in: `Materials > Session_2 > pandas_mini_task.ipynb`

*Feel free to also work on:* `Session_1 > basic_python_tasks.ipynb`

### For help: 
- <a href="https://github.com/TiesdeKok/LearnPythonforResearch">Python tutorial</a>
- <a href="https://nbviewer.jupyter.org/github/TiesdeKok/LearnPythonforResearch/blob/master/0_python_basics.ipynb">Python Basics Notebook</a>
- <a href="https://nbviewer.jupyter.org/github/TiesdeKok/LearnPythonforResearch/blob/master/1_opening_files.ipynbb">Opening files with Python / Pandas</a>
- <a href="https://nbviewer.jupyter.org/github/TiesdeKok/LearnPythonforResearch/blob/master/2_handling_data.ipynb">Data handling with Pandas</a>

]