Delphi XE2 : Off by 7-20 lines in debugger and compiler error line numbers also off by the same amount

Question

I am having a problem with a large Delphi codebase where I work where as a side effect of porting from Delphi 2007 to XE2, we now encounter the following strange, related issues:

• You can not set breakpoints or single step through code even in the debug build because the line numbering is all messed up, only in some of the units.

• Introducing a syntax error deliberately at line 2010 will cause the cursor to focus at line 2020, give or take 3 or 4 lines, something like this:

.

 procedure Correct;
 begin
    DoSomething; // syntax error reported HERE but the real error is below.
    // more stuff here.
 end;

 procedure OllKorrect;  
 begin
        ThisLineIsFine();
        __VARIABLE_NOT_DEFINED__ := 1; // intentional error
 end

I am hoping someone has seen this before. Elements of the issue may include:

The code contains many odd compiler directives like {$REALCOMPATIBILITY ON} and {$H-}/{$H+} directives, thousands of {$H+}/{$H-} directives in the code.

Secondly, the code uses a lot of {$I INCLUDE} directives, and I suspect that include files might directly mess up the line-numbering of the compiler.

I am unable to say for sure, but I suspect all these old "make it work like turbo pascal for DOS" compiler switches are the reason behind it. I'd like to know if anyone knows something about this for sure. It only happens in some places in the code, and with a project that has over 500 units, some of which reach 10K/20KLOC in size, it is definitely frustrating. What I can say is that it not only units that have {$I include.inc} directives that mess up, and that many units that contain a lot of {$H-}/{$H+} or {$REALCOMPATIBILITY} directives do not have this problem. If I could see what the units that misbehave have in common I could figure this out.

Update: The line termination issue makes sense. I ran this code which detected problems. The fix code is commented out because if you uncomment it and it erases all your source code, that's your problem. It is loading a non-unicode file into a unicode TStringList and saving it back out. That's okay in my world because its all version controlled and backed up. Your mileage may vary.

program linefeedsProject1;

{$APPTYPE CONSOLE}

uses
  IOUtils,
  Classes,
  Types,
  SysUtils;


  var
    broken,notBroken:Integer;

  function fix(filename:String):Boolean;
  var
    sl:TStringList;
  begin
    sl := TStringList.Create;
    try
    sl.LoadFromFile(filename);
    //TODO:Change file extensions.
    sl.SaveToFile(filename);
    finally
      sl.Free;
    end;
  end;

  function scan(filename:String):Boolean;
  var
  crFlag:Boolean;
  lfFlag:Boolean;
  missingCr:Integer;
  missingLf:Integer;
   f:TFileStream;
   buf:Array[0..1024] of AnsiChar;
   n:Integer;
   procedure scanChars;
   var
    i:Integer;
   begin
     for i := 0 to n-1 do
     begin
       if buf[i]=#13 then
       begin
          crFlag := true;
          lfFlag := false;
       end
       else if buf[i]=#10 then
       begin
           if not crFlag then
            inc(missingCr);
          lfFlag := true;
          crFlag := false;
       end
       else begin
         if (crFlag) then
          inc(missingLf);
         crFlag := false;
         lfFlag := false;
       end;
     end;
   end;
  begin
   result := false;
   crFlag := false;
   lfFlag := false;
   missingCr := 0;
   missingLf := 0;
    f := TFileStream.Create(filename, fmOpenRead);
    try
      while f.Position< f.Size do
      begin
        n := f.Read(buf[0],1024);
        scanChars;
      end;

     if (missingCr>0) or (missingLf>0) then
     begin
          WriteLn('  ', filename);
          Inc(broken);
          result := true;
     end
     else
     begin
        Inc(notBroken);
     end
    finally
      f.Free;
    end;

  end;
var
 files:TStringDynArray;
 afile:String;
 begin
  try
  broken := 0;
  notBroken := 0;
    files := TDirectory.GetFiles('C:\dev\abackupcopyofyoursourcecode',  '*.pas',
    TSearchOption.soTopDirectoryOnly );
    // tried TSearchOption.soAllDirectories and it exploded. not recommended.

    for afile in files do
    begin
       if scan(afile) then
       begin
           // fix(afile); // uncomment at your own risk and only on a backup copy of your code.
       end;

    end;


    WriteLn('Broken ', broken);
    WriteLn('not broken ',notBroken);

   // readln;

     except
    on E: Exception do
      Writeln(E.ClassName, ': ', E.Message);
  end;
end.

Update 2: If you want a scanner/fixer for this issue you can download mine (with source) here. Link is Google Drive. You can view the source code from the link, but click the "File" pull down menu (part of the google drive web user interface) and then click "Download" to download it.

Answer 1

I've seen things like this before, and IME it's generally due to an compiler bug in counting line numbers. If you have nonstandard line breaks (not CRLF) at some points--which can happen--the IDE will do proper line breaks but the compiler doesn't count them as new lines, so everything afterwards gets thrown off by one.

What I do when I encounter a file like this is open it in EditPad, convert all linebreaks to some other style (Unix or Mac style) and then convert all linebreaks to Windows style and save it. This ensures that every line in the file ends with CRLF, and after a rebuild the issue with the blue dots not lining up right goes away.

Answer 2

This is a common problem caused by mismatched line termination characters. (Code with a missing CR or LF on the end of the line.) It can also be caused by having a .dcu that doesn't match the source file that's open in the editor.

For the first, the easiest fix is to open the source file in a regular text editor (such as Notepad). Make a small change (insert a blank line and then delete it), and save the file. Notepad will fix the line endings.

If this isn't the issue, look for extra copies of the .dcu (or .pas) file that might be on your drive in the IDE's search path. Sometimes the compiler sees a different version than what's open in the editor.