Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Django AWS Elastic Beanstalk error ImproperlyConfigured: Error loading MySQLdb module: No module named MySQLdb

I know this error have come to many people and I have tried different solutions and none of them worked.

I am using aws eb cli.

I am using following command eb deploy to deploy my application to server.

Following are the configuration for my Django.

under .ebextensions directory, I have following 2 files:

1: 01_packages.config

packages:
  yum:
    git: []
    python27-devel: []
    mysql: []
    mysql-devel: []

and another file is

2: 02_django.conf

option_settings:
  "aws:elasticbeanstalk:application:environment":
    DJANGO_SETTINGS_MODULE: "settings.development"
    "PYTHONPATH": "/opt/python/current/app/src:$PYTHONPATH"
  "aws:elasticbeanstalk:container:python":
    WSGIPath: src/wsgi.py
    NumProcesses: 3
    NumThreads: 20
  "aws:elasticbeanstalk:container:python:staticfiles":
    "/static/": "static/"

Following is my requirements.txt file after pip freeze in my local virtual environment.

requirements.txt

asn1crypto==0.24.0
awsebcli==3.17.1
backports.ssl-match-hostname==3.5.0.1
botocore==1.14.17
cement==2.8.2
cent==2.1.0
centrifuge==0.8.4
certifi==2017.11.5
cffi==1.11.2
chardet==3.0.4
colorama==0.3.9
cryptography==2.1.4
Django==1.8.18
django-colorfield==0.1.14
django-countries==5.0
django-debug-toolbar==1.9.1
django-environ==0.4.4
django-multiselectfield==0.1.8
django-simple-history==1.9.1
django-sslserver==0.20
docutils==0.15.2
enum34==1.1.6
future==0.16.0
google-api-python-client==1.6.4
hiredis==0.2.0
html5lib==1.0b8
httplib2==0.10.3
icalendar==4.0.0
idna==2.6
ipaddress==1.0.18
jmespath==0.9.4
jsonschema==2.4.0
mysqlclient==1.4.6
oauth2client==2.0.0
oauthclient==1.0.3
olefile==0.44
pathspec==0.5.9
paypalrestsdk==1.13.1
pdfcrowd==4.0.1
phonenumbers==8.8.6
Pillow==4.3.0
pyasn1==0.3.7
pyasn1-modules==0.1.5
pycparser==2.18
PyJWT==1.5.3
pyOpenSSL==17.5.0
PyPDF2==1.26.0
pypiwin32==219
pytesseract==0.1.7
python-dateutil==2.6.1
pytz==2017.3
PyYAML==5.2
reportlab==3.4.0
requests==2.18.4
rsa==3.4.2
semantic-version==2.5.0
six==1.11.0
sockjs-tornado==1.0.1
sqlparse==0.2.4
termcolor==1.1.0
toredis-fork==0.1.4
tornado==4.2.1
toro==0.8
twilio==6.9.1
uritemplate==3.0.0
urllib3==1.22
wcwidth==0.1.8
webencodings==0.5.1
xhtml2pdf==0.2.2

I kept this in my root directory.

When I run eb deploy

it deploys successfully.

enter image description here

but when I run the browser to my url.

I get this Internal Server Error page.

enter image description here

So I tried to look in to the log files on the server under /var/log/httpd/error_log

i get the following error:

[Mon Feb 24 17:56:57.227427 2020] [:error] [pid 8054] [remote 101.50.93.65:188] mod_wsgi (pid=8054): Target WSGI script '/opt/python/current/app/src/wsgi.py' cannot be loaded as Python module.
[Mon Feb 24 17:56:57.227450 2020] [:error] [pid 8054] [remote 101.50.93.65:188] mod_wsgi (pid=8054): Exception occurred processing WSGI script '/opt/python/current/app/src/wsgi.py'.
[Mon Feb 24 17:56:57.227466 2020] [:error] [pid 8054] [remote 101.50.93.65:188] Traceback (most recent call last):
[Mon Feb 24 17:56:57.227483 2020] [:error] [pid 8054] [remote 101.50.93.65:188]   File "/opt/python/current/app/src/wsgi.py", line 17, in <module>
[Mon Feb 24 17:56:57.227585 2020] [:error] [pid 8054] [remote 101.50.93.65:188]     application = get_wsgi_application()
[Mon Feb 24 17:56:57.227599 2020] [:error] [pid 8054] [remote 101.50.93.65:188]   File "/opt/python/run/venv/local/lib/python2.7/site-packages/django/core/wsgi.py", line 14, in get_wsgi_application
[Mon Feb 24 17:56:57.227627 2020] [:error] [pid 8054] [remote 101.50.93.65:188]     django.setup()
[Mon Feb 24 17:56:57.227634 2020] [:error] [pid 8054] [remote 101.50.93.65:188]   File "/opt/python/run/venv/local/lib/python2.7/site-packages/django/__init__.py", line 18, in setup
[Mon Feb 24 17:56:57.227645 2020] [:error] [pid 8054] [remote 101.50.93.65:188]     apps.populate(settings.INSTALLED_APPS)
[Mon Feb 24 17:56:57.227651 2020] [:error] [pid 8054] [remote 101.50.93.65:188]   File "/opt/python/run/venv/local/lib/python2.7/site-packages/django/apps/registry.py", line 78, in populate
[Mon Feb 24 17:56:57.227662 2020] [:error] [pid 8054] [remote 101.50.93.65:188]     raise RuntimeError("populate() isn't reentrant")
[Mon Feb 24 17:56:57.227676 2020] [:error] [pid 8054] [remote 101.50.93.65:188] RuntimeError: populate() isn't reentrant

So I kept Googling, and found a post suggested to change wsgi file, and I did and then this error updated to the following lines of code.

[Mon Feb 24 18:23:12.169850 2020] [mpm_prefork:notice] [pid 9081] AH00169: caught SIGTERM, shutting down
[Mon Feb 24 18:23:13.289449 2020] [suexec:notice] [pid 10286] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Mon Feb 24 18:23:13.305290 2020] [so:warn] [pid 10286] AH01574: module wsgi_module is already loaded, skipping
[Mon Feb 24 18:23:13.307373 2020] [http2:warn] [pid 10286] AH10034: The mpm module (prefork.c) is not supported by mod_http2. The mpm determines how things are processed in your server. HTTP/2 has more demands in this regard and the currently selected mpm will just not do. This is an advisory warning. Your server will continue to work, but the HTTP/2 protocol will be inactive.
[Mon Feb 24 18:23:13.307384 2020] [http2:warn] [pid 10286] AH02951: mod_ssl does not seem to be enabled
[Mon Feb 24 18:23:13.307990 2020] [lbmethod_heartbeat:notice] [pid 10286] AH02282: No slotmem from mod_heartmonitor
[Mon Feb 24 18:23:13.308050 2020] [:warn] [pid 10286] mod_wsgi: Compiled for Python/2.7.13.
[Mon Feb 24 18:23:13.308057 2020] [:warn] [pid 10286] mod_wsgi: Runtime using Python/2.7.16.
[Mon Feb 24 18:23:13.311200 2020] [mpm_prefork:notice] [pid 10286] AH00163: Apache/2.4.41 (Amazon) mod_wsgi/3.5 Python/2.7.16 configured -- resuming normal operations
[Mon Feb 24 18:23:13.311217 2020] [core:notice] [pid 10286] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'
[Mon Feb 24 18:23:16.367182 2020] [:error] [pid 10293] [remote 127.0.0.1:0] mod_wsgi (pid=10293): Exception occurred processing WSGI script '/opt/python/current/app/src/wsgi.py'.
[Mon Feb 24 18:23:16.367240 2020] [:error] [pid 10293] [remote 127.0.0.1:0] RuntimeError: response has not been started
[Mon Feb 24 18:23:17.744228 2020] [:error] [pid 10291] [remote 127.0.0.1:0] mod_wsgi (pid=10291): Exception occurred processing WSGI script '/opt/python/current/app/src/wsgi.py'.
[Mon Feb 24 18:23:17.744288 2020] [:error] [pid 10291] [remote 127.0.0.1:0] RuntimeError: response has not been started
[Mon Feb 24 18:23:19.116825 2020] [:error] [pid 10292] [remote 127.0.0.1:0] mod_wsgi (pid=10292): Exception occurred processing WSGI script '/opt/python/current/app/src/wsgi.py'.
[Mon Feb 24 18:23:19.116892 2020] [:error] [pid 10292] [remote 127.0.0.1:0] RuntimeError: response has not been started
[Mon Feb 24 18:23:20.493432 2020] [:error] [pid 10418] [remote 127.0.0.1:0] mod_wsgi (pid=10418): Exception occurred processing WSGI script '/opt/python/current/app/src/wsgi.py'.
[Mon Feb 24 18:23:20.493489 2020] [:error] [pid 10418] [remote 127.0.0.1:0] RuntimeError: response has not been started
[Mon Feb 24 18:36:44.987693 2020] [:error] [pid 10443] [remote 95.105.12.68:0] mod_wsgi (pid=10443): Exception occurred processing WSGI script '/opt/python/current/app/src/wsgi.py'.
[Mon Feb 24 18:36:44.987766 2020] [:error] [pid 10443] [remote 95.105.12.68:0] RuntimeError: response has not been started
[Mon Feb 24 18:55:28.298121 2020] [:error] [pid 10468] [remote 101.50.93.65:0] mod_wsgi (pid=10468): Exception occurred processing WSGI script '/opt/python/current/app/src/wsgi.py'.
[Mon Feb 24 18:55:28.298180 2020] [:error] [pid 10468] [remote 101.50.93.65:0] RuntimeError: response has not been started
[Mon Feb 24 18:55:30.126198 2020] [:error] [pid 10499] [remote 101.50.93.65:0] mod_wsgi (pid=10499): Exception occurred processing WSGI script '/opt/python/current/app/src/wsgi.py'.
[Mon Feb 24 18:55:30.126260 2020] [:error] [pid 10499] [remote 101.50.93.65:0] RuntimeError: response has not been started
[Mon Feb 24 18:55:31.671293 2020] [:error] [pid 10973] [remote 101.50.93.65:0] mod_wsgi (pid=10973): Exception occurred processing WSGI script '/opt/python/current/app/src/wsgi.py'.
[Mon Feb 24 18:55:31.671358 2020] [:error] [pid 10973] [remote 101.50.93.65:0] RuntimeError: response has not been started
[Mon Feb 24 18:55:32.858757 2020] [:error] [pid 11606] [remote 101.50.93.65:0] mod_wsgi (pid=11606): Exception occurred processing WSGI script '/opt/python/current/app/src/wsgi.py'.
[Mon Feb 24 18:55:32.858821 2020] [:error] [pid 11606] [remote 101.50.93.65:0] RuntimeError: response has not been started

and this is the new wsgi.py file.

import os
import logging

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings.development")

from django.core.wsgi import get_wsgi_application

# application = get_wsgi_application()
env_variables_to_pass = ['DJANGO_SETTINGS_MODULE', ]


def application(environ, start_response):
    if environ['mod_wsgi.process_group'] != '':
        import signal
        os.kill(os.getpid(), signal.SIGINT)
    return ["killed"]

and following was the old wsgi.py file


import os
import logging

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings.development")

from django.core.wsgi import get_wsgi_application

application = get_wsgi_application()
env_variables_to_pass = ['DJANGO_SETTINGS_MODULE', ]

So then I tried to run application on the server using python manage.py runserver as there was nothing else to do to fix this and I was not sure where the problem is.

So I did ssh and I did used the virtualenvironment of server that was already there created by EB.

enter image description here

after running python manage.py runserver.

I get this following error.

 File "/opt/python/run/venv/local/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 27, in <module>
    raise ImproperlyConfigured("Error loading MySQLdb module: %s" % e)
django.core.exceptions.ImproperlyConfigured: Error loading MySQLdb module: No module named MySQLdb

enter image description here

So I tried to follow any advice that I could see available on google.

I tried pip install mysqlclient

I get this error.


Collecting mysqlclient
  Using cached https://files.pythonhosted.org/packages/d0/97/7326248ac8d5049968bf4ec708a5d3d4806e412a42e74160d7f266a3e03a/mysqlclient-1.4.6.tar.gz
    Complete output from command python setup.py egg_info:
    sh: mysql_config: command not found
    sh: mariadb_config: command not found
    sh: mysql_config: command not found
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-MxFUOd/mysqlclient/setup.py", line 16, in <module>
        metadata, options = get_config()
      File "setup_posix.py", line 61, in get_config
        libs = mysql_config("libs")
      File "setup_posix.py", line 29, in mysql_config
        raise EnvironmentError("%s not found" % (_mysql_config_path,))
    EnvironmentError: mysql_config not found

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-MxFUOd/mysqlclient/

I tried both

pip install mysqlclient --no-cache-dir

and

pip install mysqlclient

It's Centos server so I cannot do the sudo apt-get commands.

Only yum commands work.

I tried this following command with sudo

sudo pip install mysql-connector-python

But I think this will be installed globally rather than env.

So I tried without sudo, and it gave permission error.

I used other commands to install mysql both with sudo and not sudo.

pip install pymysql

sudo yum install python-mysqldb

No matter what I do I get this MySQL error.

I don't want to move to other database as I would have to move data as well.


UPDATE

from given suggession of @Arun K i ran this following command

which mysql_config

i got this following response.

/usr/bin/which: no mysql_config in (/opt/python/run/venv/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/aws/bin:/home/ec2-user/.local/bin:/home/ec2-user/bin)

enter image description here

like image 545
Sizzling Code Avatar asked Feb 24 '20 19:02

Sizzling Code


2 Answers

in my case on on 64bit Amazon Linux 2/3.0.1 i had to install the following via *.config inside .ebextensions

packages:
  yum:
    python3-devel: []
    mariadb-devel: []
    gcc: []
like image 188
Fadi Abo Msalam Avatar answered Nov 14 '22 08:11

Fadi Abo Msalam


Try running

sudo yum install mysql-devel gcc python-devel

Then

pip install mysqlclient
like image 2
Nafees Anwar Avatar answered Nov 14 '22 07:11

Nafees Anwar