I'm trying to run a ETL job in AWS glue using python the script is
import sys
from awsglue.transforms import *
from awsglue.utils import getResolvedOptions
from pyspark.context import SparkContext
from awsglue.context import GlueContext
from awsglue.job import Job
glueContext = GlueContext(SparkContext.getOrCreate())
person = glueContext.create_dynamic_frame.from_catalog(
database="test",
table_name="testetl_person")
person.printSchema()
This script is running in AWS development endpoint and on running the job throws the below exception
File "/tmp/runscript.py", line 118, in <module>
runpy.run_path(temp_file_path, run_name='__main__')
File "/usr/local/lib/python3.6/runpy.py", line 263, in run_path
pkg_name=pkg_name, script_name=fname)
File "/usr/local/lib/python3.6/runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "/usr/local/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/tmp/glue-python-scripts-cf4xyag5/test.py", line 2, in <module>
ModuleNotFoundError: No module named 'awsglue.transforms'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/tmp/runscript.py", line 137, in <module>
raise e_type(e_value).with_tracsback(new_stack)
AttributeError: 'ModuleNotFoundError' object has no attribute 'with_tracsback'
Can anyone help me out? If you require further information do let me know.
According to This, Your language to run the code should set as spark, not python.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With