Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

asyncio improperly warns about streams objects are garbage collected; call "stream.close()" explicitly

I was implementing asynchronous MySQL query execution using python3.8's inbuilt asyncio package and an installed aiomysql package. Even though I have closed properly all the open cursor and connection, an error message keep on appearing on my console as follows.

An open stream object is being garbage collected; call "stream.close()" explicitly.

A summary of the code is given below...

#db.py

import asyncio

class AsyncMysqlSession:

    def __init__(self, loop, db_settings=DEFAULTDB):
        self.db_settings = db_settings
        self.loop = loop

    async def __aenter__(self):
        self.conn = await aiomysql.connect(host=self.db_settings['HOST'],
                                       port=self.db_settings['PORT'],
                                       user=self.db_settings['USER'],
                                       password=self.db_settings['PASSWORD'],
                                       db=self.db_settings['NAME'],
                                       loop=self.loop)
        self.cursor = await self.conn.cursor(aiomysql.cursors.DictCursor)
        return self

    async def __aexit__(self, exception, value, traceback):
        await self.cursor.close()
        self.conn.close()

    async def query(self, sql, *args):
        await self.cursor.execute(sql, values)
        await self.conn.commit()
        rows = await self.cursor.fetchall()
        return list(rows)


async def aiomysql_query(sql, *args):
    """
    Mysql asynchronous connection wrapper
    """
    loop = asyncio.get_event_loop()
    async with AsyncMysqlSession(loop) as mysql:
        db_result = await mysql.query(sql, *args)
        return db_result

aiomysql_query is imported in another file

#views.py

 import asyncio
 .....
 

 async def main():
     .....
     .....
     
     await aiomysql_query(sql1, *args1)
     await aiomysql_query(sql2, *args2)

 .....

 asyncio.run(main())

 ....

Am I doing something wrong here (?) or is it improperly shows the error message?. Any lead to resolve this issue will be appreciated... TIA!!

like image 870
Akhil Mathew Avatar asked Sep 05 '20 09:09

Akhil Mathew


1 Answers

It seems like you may have just forgotten to close the event loop—in addition to await conn.wait_closed(), which @VPfB advised above.

You must close the event loop when manually using lower level method calls such as asyncio.get_event_loop(). Specifically, self.loop.close() must be called.

#db.py

import asyncio

class AsyncMysqlSession:

    def __init__(self, loop, db_settings=DEFAULTDB):
        self.db_settings = db_settings
        self.loop = loop

    async def __aenter__(self):
        self.conn = await aiomysql.connect(host=self.db_settings['HOST'],
                                       port=self.db_settings['PORT'],
                                       user=self.db_settings['USER'],
                                       password=self.db_settings['PASSWORD'],
                                       db=self.db_settings['NAME'],
                                       loop=self.loop)
        self.cursor = await self.conn.cursor(aiomysql.cursors.DictCursor)
        return self

    async def __aexit__(self, exception, value, traceback):
        await self.cursor.close()
        self.conn.close()
        self.loop.close()

    async def query(self, sql, *args):
        await self.cursor.execute(sql, values)
        await self.conn.commit()
        rows = await self.cursor.fetchall()
        return list(rows)


async def aiomysql_query(sql, *args):
    """
    Mysql asynchronous connection wrapper
    """
    loop = asyncio.get_event_loop()
    async with AsyncMysqlSession(loop) as mysql:
        db_result = await mysql.query(sql, *args)
        return db_result

References

https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.shutdown_asyncgens

https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.get_event_loop

like image 69
pygeek Avatar answered Oct 28 '22 22:10

pygeek