datopy.models.media#

Description

Data models, validators, and ETL tools for scraped media data, including reviews (via IMDb), music albums (via Spotify), and related information (via Wikipedia).

class Album( title: str, artist: str | None = None, )#: Bases: MediaQuery

class Book( title: str, artist: str | None = None, )#: Bases: MediaQuery

class Film( title: str, artist: str | None = None, )#: Bases: MediaQuery

pydantic model IMDbFilm[source]#

Bases: BaseModel

Data model for processed imdb metadata.

Example

>>> from pydantic import ValidationError
>>> from datopy.models.media import IMDbFilm
>>> from datopy._examples import imdb_film_retrieve

>>> valid_film = IMDbFilm(
...     title='name 10!', imdb_id='tt1234567', kind='movie',
...     year=1990, rating=7.2, votes=122,
...     genres='romantic comedy, thriller', cast='mrs smith,mr smith',
...     plot='alas! once upon a time, ...',
...     budget_mil=1123929)

>>> invalid_film = dict(
...     title='name', imdb_id='tt12', year=1975, votes=-2, rating=5.0)
>>> try:
...     IMDbFilm(**invalid_film)
... except ValidationError as e:
...     print(e)          # use pprint.pp(e.errors()) for easy-to-read list
3 validation errors for IMDbFilm
imdb_id
  String should match pattern '^tt.*\d{7}$' [type=string_pattern_mismatch, input_value='tt12', input_type=str]
    For further information visit https://errors.pydantic.dev/2.7/v/string_pattern_mismatch
kind
  Field required [type=missing, input_value={'title': 'name', 'imdb_i...tes': -2, 'rating': 5.0}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.7/v/missing
votes
  Input should be greater than or equal to 0 [type=greater_than_equal, input_value=-2, input_type=int]
    For further information visit https://errors.pydantic.dev/2.7/v/greater_than_equal

Survey available fields and types

>>> import pprint
>>> from datopy.models.media import Film
>>> from datopy._examples import imdb_film_retrieve
>>> from datopy.modeling import apply_recursive
>>> film = imdb_film_retrieve(Film('spirited away'))

Show JSON schema

{
   "title": "IMDbFilm",
   "description": "Data model for processed imdb metadata.\n\n\nExample\n-------\n>>> from pydantic import ValidationError\n>>> from datopy.models.media import IMDbFilm\n>>> from datopy._examples import imdb_film_retrieve\n\n>>> valid_film = IMDbFilm(\n...     title='name 10!', imdb_id='tt1234567', kind='movie',\n...     year=1990, rating=7.2, votes=122,\n...     genres='romantic comedy, thriller', cast='mrs smith,mr smith',\n...     plot='alas! once upon a time, ...',\n...     budget_mil=1123929)\n\n>>> invalid_film = dict(\n...     title='name', imdb_id='tt12', year=1975, votes=-2, rating=5.0)\n>>> try:\n...     IMDbFilm(**invalid_film)\n... except ValidationError as e:\n...     print(e)          # use pprint.pp(e.errors()) for easy-to-read list\n3 validation errors for IMDbFilm\nimdb_id\n  String should match pattern '^tt.*\\d{7}$' [type=string_pattern_mismatch, input_value='tt12', input_type=str]\n    For further information visit https://errors.pydantic.dev/2.7/v/string_pattern_mismatch\nkind\n  Field required [type=missing, input_value={'title': 'name', 'imdb_i...tes': -2, 'rating': 5.0}, input_type=dict]\n    For further information visit https://errors.pydantic.dev/2.7/v/missing\nvotes\n  Input should be greater than or equal to 0 [type=greater_than_equal, input_value=-2, input_type=int]\n    For further information visit https://errors.pydantic.dev/2.7/v/greater_than_equal\n\nSurvey available fields and types\n\n>>> import pprint\n>>> from datopy.models.media import Film\n>>> from datopy._examples import imdb_film_retrieve\n>>> from datopy.modeling import apply_recursive\n>>> film = imdb_film_retrieve(Film('spirited away'))\n\n..\n    # >>> film.keys()\n    # >>> pprint.pp(apply_recursive(lambda x: type(x).__name__, film), depth=3)",
   "type": "object",
   "properties": {
      "title": {
         "description": "Allows numerics",
         "pattern": "^[a-z0-9,.! ]+$",
         "title": "Title",
         "type": "string"
      },
      "imdb_id": {
         "description": "Unique 7-digit IMDb tt identifier",
         "pattern": "^tt.*\\d{7}$",
         "title": "Imdb Id",
         "type": "string"
      },
      "kind": {
         "description": "Retrieved from: `type`",
         "examples": [
            "movie",
            "tv series"
         ],
         "pattern": "^[a-z, ]+$",
         "title": "Kind",
         "type": "string"
      },
      "year": {
         "maximum": 3000,
         "minimum": 1880,
         "title": "Year",
         "type": "integer"
      },
      "rating": {
         "maximum": 10.0,
         "minimum": 0.0,
         "title": "Rating",
         "type": "number"
      },
      "votes": {
         "minimum": 0,
         "title": "Votes",
         "type": "integer"
      },
      "runtime_mins": {
         "default": null,
         "exclusiveMinimum": 0.0,
         "title": "Runtime Mins",
         "type": "number"
      },
      "genres": {
         "default": null,
         "description": "Custom lowercase comma-separated string type. Excludes num and special chars",
         "pattern": "^[a-z, ]+$",
         "title": "Genres",
         "type": "string"
      },
      "countries": {
         "default": null,
         "description": "Custom lowercase comma-separated string type. Excludes num and special chars",
         "pattern": "^[a-z, ]+$",
         "title": "Countries",
         "type": "string"
      },
      "director": {
         "default": null,
         "description": "Custom lowercase comma-separated string type. Excludes num and special chars",
         "pattern": "^[a-z, ]+$",
         "title": "Director",
         "type": "string"
      },
      "writer": {
         "default": null,
         "description": "Custom lowercase comma-separated string type. Excludes num and special chars",
         "pattern": "^[a-z, ]+$",
         "title": "Writer",
         "type": "string"
      },
      "composer": {
         "default": null,
         "description": "Custom lowercase comma-separated string type. Excludes num and special chars",
         "pattern": "^[a-z, ]+$",
         "title": "Composer",
         "type": "string"
      },
      "cast": {
         "default": null,
         "description": "Custom lowercase comma-separated string type. Excludes num and special chars",
         "pattern": "^[a-z, ]+$",
         "title": "Cast",
         "type": "string"
      },
      "plot": {
         "default": null,
         "pattern": "^[a-z0-9,.! ]+$",
         "title": "Plot",
         "type": "string"
      },
      "synopsis": {
         "default": null,
         "pattern": "^[a-z0-9,.! ]+$",
         "title": "Synopsis",
         "type": "string"
      },
      "plot_outline": {
         "default": null,
         "pattern": "^[a-z0-9,.! ]+$",
         "title": "Plot Outline",
         "type": "string"
      },
      "budget_mil": {
         "default": null,
         "description": "Strip $/, & text after first space",
         "minimum": 0.0,
         "title": "Budget Mil",
         "type": "number"
      },
      "opening_weekend_gross_mil": {
         "default": null,
         "minimum": 0.0,
         "title": "Opening Weekend Gross Mil",
         "type": "number"
      },
      "cumulative_worldwide_gross_mil": {
         "default": null,
         "minimum": 0.0,
         "title": "Cumulative Worldwide Gross Mil",
         "type": "number"
      }
   },
   "required": [
      "title",
      "imdb_id",
      "kind",
      "year",
      "rating",
      "votes"
   ]
}

field budget_mil: float = None#

Strip $/, & text after first space

Constraints:

ge = 0

field cast: str = None#

Custom lowercase comma-separated string type. Excludes num and special chars

Constraints:

pattern = ^[a-z, ]+$

field composer: str = None#

Custom lowercase comma-separated string type. Excludes num and special chars

Constraints:

pattern = ^[a-z, ]+$

field countries: str = None#

Custom lowercase comma-separated string type. Excludes num and special chars

Constraints:

pattern = ^[a-z, ]+$

field cumulative_worldwide_gross_mil: float = None#

Constraints:

ge = 0

field director: str = None#

Custom lowercase comma-separated string type. Excludes num and special chars

Constraints:

pattern = ^[a-z, ]+$

field genres: str = None#

Custom lowercase comma-separated string type. Excludes num and special chars

Constraints:

pattern = ^[a-z, ]+$

field imdb_id: str [Required]#

Unique 7-digit IMDb tt identifier

Constraints:

pattern = ^tt.*d{7}$

field kind: str [Required]#

Retrieved from: type

Constraints:

pattern = ^[a-z, ]+$

field opening_weekend_gross_mil: float = None#

Constraints:

ge = 0

field plot: str = None#

Constraints:

pattern = ^[a-z0-9,.! ]+$

field plot_outline: str = None#

Constraints:

pattern = ^[a-z0-9,.! ]+$

field rating: float [Required]#

Constraints:

ge = 0
le = 10

field runtime_mins: float = None#

Constraints:

gt = 0

field synopsis: str = None#

Constraints:

pattern = ^[a-z0-9,.! ]+$

field title: str [Required]#

Allows numerics

Constraints:

pattern = ^[a-z0-9,.! ]+$

field votes: int [Required]#

Constraints:

ge = 0

field writer: str = None#

Custom lowercase comma-separated string type. Excludes num and special chars

Constraints:

pattern = ^[a-z, ]+$

field year: int [Required]#

Constraints:

ge = 1880
le = 3000

class IMDbFilmProcessor( model: BaseModel, query: NamedTuple, )[source]#

Bases: BaseProcessor

Methods

`process`()	Process (extract/clean) retrieved data.
`retrieve`()	Retrieve data for the query from the API of the supplied model.

process()[source]#

Process (extract/clean) retrieved data.

Raises:: NotImplementedError – _description_:

retrieve()[source]#

Retrieve data for the query from the API of the supplied model.

Raises:: NotImplementedError – _description_:

class MediaQuery( title: str, artist: str | None = None, )[source]#

Bases: NamedTuple

Query object types for media metadata retrieval.

artist: str | None#: Alias for field number 1

title: str#: Alias for field number 0

pydantic model SpotifyAlbum[source]#

Bases: BaseModel

Data model for processed Spotify metadata. Raw data schema reference: ‘datopy/output/spotify_album_schema.json’

Show JSON schema

{
   "title": "SpotifyAlbum",
   "description": "Data model for processed Spotify metadata.\nRaw data schema reference: 'datopy/output/spotify_album_schema.json'",
   "type": "object",
   "properties": {
      "title": {
         "title": "Title",
         "type": "string"
      },
      "album_type": {
         "title": "Album Type",
         "type": "string"
      }
   },
   "required": [
      "title",
      "album_type"
   ]
}

field album_type: str [Required]#

field title: str [Required]#

pydantic model WikiAlbum[source]#

Bases: BaseModel

Data model for processed Wikipedia album metadata. Raw data schema reference: ‘datopy/output/wiki_album_schema.json’

Show JSON schema

{
   "title": "WikiAlbum",
   "description": "Data model for processed Wikipedia album metadata.\nRaw data schema reference: 'datopy/output/wiki_album_schema.json'",
   "type": "object",
   "properties": {
      "title": {
         "title": "Title",
         "type": "string"
      }
   },
   "required": [
      "title"
   ]
}

field title: str [Required]#

pydantic model WikiBook[source]#

Bases: BaseModel

Data model for processed Wikipedia novel metadata. Raw data schema reference: ‘datopy/output/wiki_book_schema.json’

Show JSON schema

{
   "title": "WikiBook",
   "description": "Data model for processed Wikipedia novel metadata.\nRaw data schema reference: 'datopy/output/wiki_book_schema.json'",
   "type": "object",
   "properties": {
      "title": {
         "title": "Title",
         "type": "string"
      }
   },
   "required": [
      "title"
   ]
}

field title: str [Required]#

pydantic model WikiFilm[source]#

Bases: BaseModel

Data model for processed Wikipedia film metadata. Raw data schema reference: ‘datopy/output/wiki_film_schema.json’

Show JSON schema

{
   "title": "WikiFilm",
   "description": "Data model for processed Wikipedia film metadata.\nRaw data schema reference: 'datopy/output/wiki_film_schema.json'",
   "type": "object",
   "properties": {
      "title": {
         "title": "Title",
         "type": "string"
      }
   },
   "required": [
      "title"
   ]
}

field title: str [Required]#

Classes

`Album`(title[, artist])
`Book`(title[, artist])
`Film`(title[, artist])
`IMDbFilmProcessor`(model, query)	Methods
`MediaQuery`(title[, artist])	Query object types for media metadata retrieval.