datopy: Python tools for data#

Python tools for data retrieval, I/O, and Jupyter notebook workflows.

Note

This project is under active development.

Getting Started#

To use data-tools, first install it using pip:

(.venv) $ pip install "git+https://github.com/bainmatt/data-tools.git#egg=datopy"

Clone the repo:

$ git clone https://github.com/bainmatt/datopy.git
$ cd datopy

Install dependencies:

$ conda env create -f environment.yml
$ conda activate dato-py

TODO add this section

TODO finish this section

Produce multiple parallel, informative displays of Pandas data frames and NumPy arrays for data exploration and inspection.

Example

>>> import numpy as np
>>> import pandas as pd
>>> from datopy.inspection import display, make_df

>>> df1 = make_df('AB', [1, 2]); df2 = make_df('AB', [3, 4])
>>> display('df1', 'df2', 'pd.concat([df1, df2])', globs=globals(), bold=False)

df1
--- (2, 2) ---
   A   B
1  A1  B1
2  A2  B2


df2
--- (2, 2) ---
   A   B
3  A3  B3
4  A4  B4


pd.concat([df1, df2])
--- (4, 2) ---
   A   B
1  A1  B1
2  A2  B2
3  A3  B3
4  A4  B4

x: Use nb_utils.py to save your Colab environment files to your mounted Google Drive from within a Colab notebook.

Media scraping: Use media_scrape.py to scrape media-related data from Spotify, IMDb, and Wikipedia.

This project is licensed under the MIT License.