Tokenization is skipped for long lines for performance reasons. The length of a long line can be configured via `editor.maxTokenizationLineLength`

Question

I'm at a loss here. I have a Node.js server built for my project and out of nowhere, I started getting this error when I try to run the server in VSCode:

Tokenization is skipped for long lines for performance reasons. The length of a long line can be configured via editor.maxTokenizationLineLength.

I had edited some of my functions in my controller before this started. I discarded all changes but the error still persists. I don't understand what has changed for this to happen. Any advice would be greatly appreciated.

My code is here if you want to take a look https://github.com/Kevinclane/timeclock

Answer 1

I fixed this problem by increasing Max Tokenization Length in VSCode :

1. Go to File--> Preferences--> Setting
2. Search for Max Tokenization Length
3. Change the default value to 2000000

I hope this will help

Answer 2

I found the hard way that besides increasing the Max Tokenization Length as suggested by @Ousama, in order to avoid VS Code from crashing or becoming entirely unresponsive you might also want to enable the Async Tokenization option in the settings. Otherwise files with really long lines will make VS Code unresponsive. I'm not sure why this was not made default.

• GitHub issue introducing Async Tokenization.