How can I effectively test against the Windows API?

Question

I'm still having issues justifying TDD to myself. As I have mentioned in other questions, 90% of the code I write does absolutely nothing but

1. Call some Windows API functions and
2. Print out the data returned from said functions.

The time spent coming up with the fake data that the code needs to process under TDD is incredible -- I literally spend 5 times as much time coming up with the example data as I would spend just writing application code.

Part of this problem is that often I'm programming against APIs with which I have little experience, which forces me to write small applications that show me how the real API behaves so that I can write effective fakes/mocks on top of that API. Writing implementation first is the opposite of TDD, but in this case it is unavoidable: I do not know how the real API behaves, so how on earth am I going to be able to create a fake implementation of the API without playing with it?

I have read several books on the subject, including Kent Beck's Test Driven Development, By Example, and Michael Feathers' Working Effectively with Legacy Code, which seem to be gospel for TDD fanatics. Feathers' book comes close in the way it describes breaking out dependencies, but even then, the examples provided have one thing in common:

• The program under test obtains input from other parts of the program under test.

My programs do not follow that pattern. Instead, the only input to the program itself is the system upon which it runs.

How can one effectively employ TDD on such a project? I'm already wrapping most of the API inside C++ classes before I actually use that API, but sometimes the wrappers themselves can become quite complicated, and deserve their own tests.

Answer 1

See below for FindFirstFile/FindNextFile/FindClose example

---

I use googlemock. For an external API I generally create an interface class. Assume I was going to call fopen, fwrite, fclose

class FileIOInterface {
public:
  ~virtual FileIOInterface() {}

  virtual FILE* Open(const char* filename, const char* mode) = 0;
  virtual size_t Write(const void* data, size_t size, size_t num, FILE* file) = 0;
  virtual int Close(FILE* file) = 0;
};

The actual implementation would be this

class FileIO : public FileIOInterface {
public:
  virtual FILE* Open(const char* filename, const char* mode) {
    return fopen(filename, mode);
  }

  virtual size_t Write(const void* data, size_t size, size_t num, FILE* file) {
    return fwrite(data, size, num, file);
  }

  virtual int Close(FILE* file) {
    return fclose(file);
  }
};

Then using googlemock I make a MockFileIO class like this

class MockFileIO : public FileIOInterface {
public:
  virtual ~MockFileIO() { }

  MOCK_MEHTOD2(Open, FILE*(const char* filename, const char* mode));
  MOCK_METHOD4(Write, size_t(const void* data, size_t size, size_t num, FILE* file));
  MOCK_METHOD1(Close, int(FILE* file));
}

This makes writing the tests easy. I don't have to provide a test implementation of Open/Write/Close. googlemock handles that for me. as in. (note I use googletest for my unit testing framework.)

Assume I have a function like this that needs testing

// Writes a file, returns true on success.
bool WriteFile(FileIOInterface fio, const char* filename, const void* data, size_size) {
   FILE* file = fio.Open(filename, "wb");
   if (!file) {
     return false;
   }

   if (fio.Write(data, 1, size, file) != size) {
     return false;
   }

   if (fio.Close(file) != 0) {
     return false;
   }

   return true;
}

And here's the tests.

TEST(WriteFileTest, SuccessWorks) {
  MockFileIO fio;

  static char data[] = "hello";
  const char* kName = "test";
  File test_file;

  // Tell the mock to expect certain calls and what to 
  // return on those calls.
  EXPECT_CALL(fio, Open(kName, "wb")
      .WillOnce(Return(&test_file));
  EXPECT_CALL(fio, Write(&data, 1, sizeof(data), &test_file))
      .WillOnce(Return(sizeof(data)));
  EXPECT_CALL(file, Close(&test_file))
      .WillOnce(Return(0));

  EXPECT_TRUE(WriteFile(kName, &data, sizeof(data));
}

TEST(WriteFileTest, FailsIfOpenFails) {
  MockFileIO fio;

  static char data[] = "hello";
  const char* kName = "test";
  File test_file;

  // Tell the mock to expect certain calls and what to 
  // return on those calls.
  EXPECT_CALL(fio, Open(kName, "wb")
      .WillOnce(Return(NULL));

  EXPECT_FALSE(WriteFile(kName, &data, sizeof(data));
}

TEST(WriteFileTest, FailsIfWriteFails) {
  MockFileIO fio;

  static char data[] = "hello";
  const char* kName = "test";
  File test_file;

  // Tell the mock to expect certain calls and what to 
  // return on those calls.
  EXPECT_CALL(fio, Open(kName, "wb")
      .WillOnce(Return(&test_file));
  EXPECT_CALL(fio, Write(&data, 1, sizeof(data), &test_file))
      .WillOnce(Return(0));

  EXPECT_FALSE(WriteFile(kName, &data, sizeof(data));
}

TEST(WriteFileTest, FailsIfCloseFails) {
  MockFileIO fio;

  static char data[] = "hello";
  const char* kName = "test";
  File test_file;

  // Tell the mock to expect certain calls and what to 
  // return on those calls.
  EXPECT_CALL(fio, Open(kName, "wb")
      .WillOnce(Return(&test_file));
  EXPECT_CALL(fio, Write(&data, 1, sizeof(data), &test_file))
      .WillOnce(Return(sizeof(data)));
  EXPECT_CALL(file, Close(&test_file))
      .WillOnce(Return(EOF));

  EXPECT_FALSE(WriteFile(kName, &data, sizeof(data));
}

I didn't have to provide a test implementation of fopen/fwrite/fclose. googlemock handles this for me. You can make the mock strict if you want. A Strict mock will fail the tests if any function that is not expected is called or if any function that is expected is called with the wrong arguments. Googlemock provides a ton of helpers and adapters so you generally don't need to write much code to get the mock to do what you want. It takes a few days to learn the different adapters but if you're using it often they quickly become second nature.

---

Here's an example using FindFirstFile, FindNextFile, FindClose

First the interface

class FindFileInterface {
public:
  virtual HANDLE FindFirstFile(
    LPCTSTR lpFileName,
    LPWIN32_FIND_DATA lpFindFileData) = 0;

  virtual BOOL FindNextFile(
    HANDLE hFindFile,
    LPWIN32_FIND_DATA lpFindFileData) = 0;

  virtual BOOL FindClose(
    HANDLE hFindFile) = 0;

  virtual DWORD GetLastError(void) = 0;
};

Then the actual implementation

class FindFileImpl : public FindFileInterface {
public:
  virtual HANDLE FindFirstFile(
    LPCTSTR lpFileName,
    LPWIN32_FIND_DATA lpFindFileData) {
    return ::FindFirstFile(lpFileName, lpFindFileData);
  }

  virtual BOOL FindNextFile(
    HANDLE hFindFile,
    LPWIN32_FIND_DATA lpFindFileData) {
    return ::FindNextFile(hFindFile, lpFindFileData);
  }

  virtual BOOL FindClose(
    HANDLE hFindFile) {
    return ::FindClose(hFindFile);
  }

  virtual DWORD GetLastError(void) {
    return ::GetLastError();
  }
};

The Mock using gmock

class MockFindFile : public FindFileInterface {
public:
  MOCK_METHOD2(FindFirstFile,
               HANDLE(LPCTSTR lpFileName, LPWIN32_FIND_DATA lpFindFileData));
  MOCK_METHOD2(FindNextFile,
               BOOL(HANDLE hFindFile, LPWIN32_FIND_DATA lpFindFileData));
  MOCK_METHOD1(FindClose, BOOL(HANDLE hFindFile));
  MOCK_METHOD0(GetLastError, DWORD());
};

The function I need to test.

DWORD PrintListing(FindFileInterface* findFile, const TCHAR* path) {
  WIN32_FIND_DATA ffd;
  HANDLE hFind;

  hFind = findFile->FindFirstFile(path, &ffd);
  if (hFind == INVALID_HANDLE_VALUE)
  {
     printf ("FindFirstFile failed");
     return 0;
  }

  do {
    if (ffd.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) {
       _tprintf(TEXT("  %s   <DIR>\n"), ffd.cFileName);
    } else {
      LARGE_INTEGER filesize;
      filesize.LowPart = ffd.nFileSizeLow;
      filesize.HighPart = ffd.nFileSizeHigh;
      _tprintf(TEXT("  %s   %ld bytes\n"), ffd.cFileName, filesize.QuadPart);
    }
  } while(findFile->FindNextFile(hFind, &ffd) != 0);

  DWORD dwError = findFile->GetLastError();
  if (dwError != ERROR_NO_MORE_FILES) {
    _tprintf(TEXT("error %d"), dwError);
  }

  findFile->FindClose(hFind);
  return dwError;
}

The unit tests.

#include <gtest/gtest.h>
#include <gmock/gmock.h>

using ::testing::_;
using ::testing::Return;
using ::testing::DoAll;
using ::testing::SetArgumentPointee;

// Some data for unit tests.
static WIN32_FIND_DATA File1 = {
  FILE_ATTRIBUTE_NORMAL,  // DWORD    dwFileAttributes;
  { 123, 0, },            // FILETIME ftCreationTime;
  { 123, 0, },            // FILETIME ftLastAccessTime;
  { 123, 0, },            // FILETIME ftLastWriteTime;
  0,                      // DWORD    nFileSizeHigh;
  123,                    // DWORD    nFileSizeLow;
  0,                      // DWORD    dwReserved0;
  0,                      // DWORD    dwReserved1;
  { TEXT("foo.txt") },    // TCHAR   cFileName[MAX_PATH];
  { TEXT("foo.txt") },    // TCHAR    cAlternateFileName[14];
};

static WIN32_FIND_DATA Dir1 = {
  FILE_ATTRIBUTE_DIRECTORY,  // DWORD    dwFileAttributes;
  { 123, 0, },            // FILETIME ftCreationTime;
  { 123, 0, },            // FILETIME ftLastAccessTime;
  { 123, 0, },            // FILETIME ftLastWriteTime;
  0,                      // DWORD    nFileSizeHigh;
  123,                    // DWORD    nFileSizeLow;
  0,                      // DWORD    dwReserved0;
  0,                      // DWORD    dwReserved1;
  { TEXT("foo.dir") },    // TCHAR   cFileName[MAX_PATH];
  { TEXT("foo.dir") },    // TCHAR    cAlternateFileName[14];
};

TEST(PrintListingTest, TwoFiles) {
  const TCHAR* kPath = TEXT("c:\\*");
  const HANDLE kValidHandle = reinterpret_cast<HANDLE>(1234);
  MockFindFile ff;

  EXPECT_CALL(ff, FindFirstFile(kPath, _))
    .WillOnce(DoAll(SetArgumentPointee<1>(Dir1),
                    Return(kValidHandle)));
  EXPECT_CALL(ff, FindNextFile(kValidHandle, _))
    .WillOnce(DoAll(SetArgumentPointee<1>(File1),
                    Return(TRUE)))
    .WillOnce(Return(FALSE));
  EXPECT_CALL(ff, GetLastError())
    .WillOnce(Return(ERROR_NO_MORE_FILES));
  EXPECT_CALL(ff, FindClose(kValidHandle));

  PrintListing(&ff, kPath);
}

TEST(PrintListingTest, OneFile) {
  const TCHAR* kPath = TEXT("c:\\*");
  const HANDLE kValidHandle = reinterpret_cast<HANDLE>(1234);
  MockFindFile ff;

  EXPECT_CALL(ff, FindFirstFile(kPath, _))
    .WillOnce(DoAll(SetArgumentPointee<1>(Dir1),
                    Return(kValidHandle)));
  EXPECT_CALL(ff, FindNextFile(kValidHandle, _))
    .WillOnce(Return(FALSE));
  EXPECT_CALL(ff, GetLastError())
    .WillOnce(Return(ERROR_NO_MORE_FILES));
  EXPECT_CALL(ff, FindClose(kValidHandle));

  PrintListing(&ff, kPath);
}

TEST(PrintListingTest, ZeroFiles) {
  const TCHAR* kPath = TEXT("c:\\*");
  MockFindFile ff;

  EXPECT_CALL(ff, FindFirstFile(kPath, _))
    .WillOnce(Return(INVALID_HANDLE_VALUE));

  PrintListing(&ff, kPath);
}

TEST(PrintListingTest, Error) {
  const TCHAR* kPath = TEXT("c:\\*");
  const HANDLE kValidHandle = reinterpret_cast<HANDLE>(1234);
  MockFindFile ff;

  EXPECT_CALL(ff, FindFirstFile(kPath, _))
    .WillOnce(DoAll(SetArgumentPointee<1>(Dir1),
                    Return(kValidHandle)));
  EXPECT_CALL(ff, FindNextFile(kValidHandle, _))
    .WillOnce(Return(FALSE));
  EXPECT_CALL(ff, GetLastError())
    .WillOnce(Return(ERROR_ACCESS_DENIED));
  EXPECT_CALL(ff, FindClose(kValidHandle));

  PrintListing(&ff, kPath);
}

I didn't have to implement any of the mock functions.

Answer 2

I don't think it's feasible to unit test thin wrapper classes. The thicker your wrapper, the easier it should be to test the bits that don't directly hit the API, as the wrapper itself can have multiple layers, the lowest of which can be mocked somehow.

While you could do something like:

// assuming Windows, sorry.

namespace Wrapper
{
   std::string GetComputerName()
   {
      char name[MAX_CNAME_OR_SOMETHING];
      ::GetComputerName(name);
      return std::string(name);
   }
}

TEST(GetComputerName) // UnitTest++
{
   CHECK_EQUAL(std::string(getenv("COMPUTERNAME")), Wrapper::GetComputerName());
}

I don't know that tests like this bring a lot of value, and would tend to make my testing focus on the transformation of data rather than the collection of such.