OpenEnergy Platform

OpenEnergyPlatformPermalink



Usage of the oedialect a python sqlalchemy dialect for the OEPPermalink

Repository: https://github.com/openego/oedialect
Documentation: http://oep-data-interface.readthedocs.io/en/latest/api/how_to.html

Please report bugs and improvements here: https://github.com/OpenEnergyPlatform/oedialect/issues

In [ ]:
__copyright__ = "Reiner Lemoine Institut"
__license__   = "GNU Affero General Public License Version 3 (AGPL-3.0)"
__url__       = "https://github.com/openego/data_processing/blob/master/LICENSE"
__author__    = "henhuy, Ludee, oakca"
In [ ]:
import pandas as pd
import getpass
import sqlalchemy as sa
from sqlalchemy.orm import sessionmaker
import oedialect

Connection to OEPPermalink

If we want to upload data to the OEP we first need to connect to it, using our OEP user name and token. Note: You ca view your token on your OEP profile page after logging in.

In [ ]:
# Whitespaces are not a problem for setting up the url!
user = input('Enter OEP-username:')
token = getpass.getpass('Token:')
In [ ]:
# Create Engine:
OEP_URL = 'oep.iks.cs.ovgu.de'
OED_STRING = f'postgresql+oedialect://{user}:{token}@{OEP_URL}'

engine = sa.create_engine(OED_STRING)
metadata = sa.MetaData(bind=engine)

Setup a TablePermalink

We need to tell the data base what columns and datatypes we are about to upload. In our case we have four columns, two of which are text, one is integer and the last is float.

In [ ]:
table_name = 'example_dialect_tablex'
schema_name = 'sandbox'

ExampleTable = sa.Table(
    table_name,
    metadata,
    sa.Column('name', sa.VARCHAR(50)),
    sa.Column('age', sa.INTEGER),
    sa.Column('stadtname', sa.VARCHAR(50)),
    schema=schema_name
)

Create the new TablePermalink

Now we tell our engine to connect to the data base and create the defined table within the chosen schema.

In [ ]:
conn = engine.connect()
print('Connection established')
if not engine.dialect.has_table(conn, table_name, schema_name):
    ExampleTable.create()
    print('Created table')
else:
    print('Table already exists')

Insert data into TablePermalink

Uploading the information from our DataFrame is now done with a single command. Uploading data in this way will always delete the content of the table and refill it with new values every time. If you change 'replace' to 'append', the data entries will be added to the preexisting ones. (Connecting and uploading may take a minute.)

In [ ]:
Session = sessionmaker(bind=engine)
session = Session()
try:
    insert_statement = ExampleTable.insert().values(
        [
            dict(name='Carsten', age=25, stadtname='Berlin'),
            dict(name='Bert', age=42, stadtname='Hamburg'),
            dict(name='Rhamses', age=69, stadtname='M√ľnchen')
        ]
    )
    session.execute(insert_statement)
    session.commit()
    print('Insert successful!')
except Exception as e:
    session.rollback()
    raise
    print('Insert incomplete!')
finally:
    session.close()

Select from TablePermalink

Now we can query our table to see if the data arrived.

In [ ]:
Session = sessionmaker(bind=engine)
session = Session()
print(session.query(ExampleTable).all())
session.close()
In [ ]:
Session = sessionmaker(bind=engine)
session = Session()
df = pd.DataFrame(session.query(ExampleTable).all())
session.close()
df
In [ ]: