I have the following code in main.py, which is a simplified version of my project where I retrieve some data, clean it and export it to a SQL database in Azure.
The problem is that when I run it locally with Azure functions everything works okay and I can see the logging in my terminal.
But when I run it with azure start --verbose, the function starts correctly, but the execution my function export_data does not work.
Simplified code:
# import packages
# load_dotenv()
def get_data():
# function which retrieves data
def export_data():
# function which exports data and uses environment variables
def main(timer: func.TimerRequest) -> None:
utc_timestamp = datetime.datetime.utcnow().replace(
tzinfo=datetime.timezone.utc).isoformat()
if timer.past_due:
logging.info('The timer is past due! Writing data to database')
export_data()
logging.info('Python timer trigger function ran at %s', utc_timestamp)
if __name__ == "__main__":
export_data()
My code:
import os
import datetime
import logging
import pandas as pd
from sqlalchemy import create_engine
from dotenv import load_dotenv
import azure.functions as func
load_dotenv()
logging.basicConfig(
format="%(asctime)s.%(msecs)03d [%(levelname)-5s] [%(name)s] - %(message)s",
datefmt="%Y-%m-%d %H:%M:%S",
level=logging.INFO,
)
def get_data():
# get data
logging.info("retrieving data")
url = "https://data.rivm.nl/covid-19/COVID-19_aantallen_gemeente_per_dag.csv"
df = pd.read_csv(url, sep=";")
return df
def export_data():
# export data
df = get_data()
uid = os.environ.get("username")
password = os.environ.get("pw")
server = '***.database.windows.net'
database = os.environ.get("database")
driver = "ODBC Driver 17 for SQL Server"
connection_string = (
f"mssql+pyodbc://{uid}:{password}@" f"{server}:1433/{database}?driver={driver}"
)
engine = create_engine(connection_string)
con = engine.connect()
logging.info("exporting data")
df.head(10).to_sql('covid_19', con=con, if_exists='replace')
def main(timer: func.TimerRequest) -> None:
utc_timestamp = datetime.datetime.utcnow().replace(
tzinfo=datetime.timezone.utc).isoformat()
if timer.past_due:
logging.info('The timer is past due! Writing data to database')
export_data()
logging.info('Python timer trigger function ran at %s', utc_timestamp)
if __name__ == "__main__":
export_data()
function.json:
{
"scriptFile": "main.py",
"bindings": [
{
"name": "timer",
"type": "timerTrigger",
"direction": "in",
"schedule": "0 */5 * * * *"
}
]
}
Edit
So here's the part of the log which shows the functions runs when I initially evoke the function with func start --verbose. Notice the time, it does not happen on the whole 5 minutes, as my schedule is based on that ("schedule": "0 */5 * * * *"):
[2020-10-23T09:23:43.339] The timer is past due! Writing data to database
[2020-10-23T09:23:43.340] retrieving data
[2020-10-23T09:23:45.323] exporting data
[2020-10-23T09:23:46.182] Python timer trigger function ran at 2020-10-23T09:23:43.332697+00:00
Then after this part, the function will run based on the timer trigger, but nothing happens:
[2020-10-23T09:25:00.009] Executing 'Functions.zyppcovid' (Reason='Timer fired at 2020-10-23T11:25:00.0076610+02:00', Id=***)
[2020-10-23T09:25:00.012] Received FunctionInvocationRequest, request ID: ***, function ID: ***, invocation ID: ***
[2020-10-23T09:25:00.013] Function is sync, request ID: ***,function ID: ***, invocation ID: ***
[2020-10-23T09:25:00.013] Python timer trigger function ran at 2020-10-23T09:25:00.012563+00:00
[2020-10-23T09:25:00.013] Successfully processed FunctionInvocationRequest, request ID: ***, function ID: ***, invocation ID: ***
[2020-10-23T09:25:00.014] Executed 'Functions.zyppcovid' (Succeeded, Id=***, Duration=6ms)
The code already have a main function for timer trigger def main(timer: func.TimerRequest) -> None:. So it can't do another main if __name__ == "__main__":. I test it in my side, if I just create a simple python code with main, it can do export_data() under the main function success.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With