--- title: Data Loader keywords: fastai sidebar: home_sidebar summary: "Generic data ingestion routines to ingest data from files to databases." description: "Generic data ingestion routines to ingest data from files to databases." nb_path: "01_data.loader.ipynb" ---
ObjectFactor
, DbSinkProvider
and FileSourceProvider
are the factory classes.
DatabaseType
and FileSource
classes.
excel_source = file_sources.get(FileSource.Excel, file_path="data/accounts.xlsx")
excel_source.get_data()
csv_source = file_sources.get(FileSource.CSV, file_path="data/accounts.csv")
csv_source.get_data()
config = {
'host': 'localhost',
'port': 5432,
'db': 'testdb',
'user': 'user1',
'password': 'userpwd'
}
pgsql_target = db_targets.get(DatabaseTarget.PostgreSQL, **config)
pgsql_target.get_conn_str()
ingest(excel_source, pgsql_target, 'accounts')
ingest(excel_source, pgsql_target, 'accounts', if_exists='replace')
config = {
'host': 'localhost',
'port': 3306,
'db': 'testdb',
'user': 'user1',
'password': 'userpwd'
}
mysql_target = db_targets.get(DatabaseTarget.MySQL, **config)
mysql_target.get_conn_str()
ingest(excel_source, mysql_target, 'accounts')
ingest(excel_source, mysql_target, 'accounts', if_exists='replace')