Debugging Scrapy Project in Visual Studio Code

Question

I have Visual Studio Code on a Windows Machine, on which I am making a new Scrapy Crawler. The crawler is working fine but I want to debug the code, for which I am adding this in my launch.json file:

{     "name": "Scrapy with Integrated Terminal/Console",     "type": "python",     "request": "launch",     "stopOnEntry": true,     "pythonPath": "${config:python.pythonPath}",     "program": "C:/Users/neo/.virtualenvs/Gers-Crawler-77pVkqzP/Scripts/scrapy.exe",     "cwd": "${workspaceRoot}",     "args": [         "crawl",         "amazon",         "-o",         "amazon.json"     ],     "console": "integratedTerminal",     "env": {},     "envFile": "${workspaceRoot}/.env",     "debugOptions": [         "RedirectOutput"     ] } 

But I am unable to hit any breakpoints. PS: I took the JSON script from here: http://www.stevetrefethen.com/blog/debugging-a-python-scrapy-project-in-vscode

Answer 1

In order to execute the typical scrapy runspider <PYTHON_FILE> command you must to set the following config into your launch.json:

{     "version": "0.1.0",     "configurations": [         {             "name": "Python: Launch Scrapy Spider",             "type": "python",             "request": "launch",             "module": "scrapy",             "args": [                 "runspider",                 "${file}"             ],             "console": "integratedTerminal"         }     ] } 

Set the breakpoints wherever you want and then debug.