Difference in std::regex_replace behaviour between GCC and MSVC

Question

I'm trying to implement a method to escape a string for use with a regex match.

Unfortunately, I'm finding an inconsistency between compilers. This code (with added boost implementation for comparison) produces different results when compiled with GCC 7.1 and Visual Studio 2015U3:

#include <iostream>
#include <regex>
#include <string>
#include <boost/regex.hpp>

std::string regexEscape(const std::string& s)
{
    return std::regex_replace(s, std::regex{ R"([\^\.\$\|\{\}\(\)\[\]\*\+\?\/\\])" }, std::string{ R"(\\\1&)" }, std::regex_constants::match_default | std::regex_constants::format_sed);
}

std::string boostRegexEscape(const std::string& s)
{
    return boost::regex_replace(s, boost::regex{ R"([\^\.\$\|\{\}\(\)\[\]\*\+\?\/\\])" }, std::string{ R"(\\\1&)" }, boost::match_default | boost::format_sed);
}

int main()
{
    std::string test{ R"(123.456^789$123\456|789*123+456(789)123?456)" };
    std::cout << regexEscape(test) << '\n';
    std::cout << boostRegexEscape(test) << '\n';
}

GCC:

123\\.456\\^789\\$123\\\456\\|789\\*123\\+456\\(789\\)123\\?456
123\.456\^789\$123\\456\|789\*123\+456\(789\)123\?456

MSVC:

123\.456\^789\$123\\456\|789\*123\+456\(789\)123\?456
123\.456\^789\$123\\456\|789\*123\+456\(789\)123\?456

Is this expected behaviour?

Answer 1

You asked the regex engines to replace with R"(\\\1&)", a \\\1& substitution pattern treating it as a sed replacement pattern. In sed, & stands for the whole match. As there is no group with ID 1 in the pattern the \1 refers to an empty string. The first two backslashes are 2 literal backslashes in a raw string literal when parsed with the std::regex_replace.

When you use Boost, the first two backslashes are parsed as a single backslash, a literal backslash in a Boost replacement pattern must be escaped in order to use a single literal backslash as a replacement:

Sed-style format strings treat all characters as literals except:

& The ampersand character is replaced in the output stream by the the whole of what matched the regular expression. Use \& to output a literal '&' character.

\ Specifies an escape sequence.

Regarding the rest of the replacement pattern, it will work the same.

You may use

 std::regex_replace(s, std::regex{ R"(([.^$|{}()[\]*+?/\\]))" }, std::string{ R"(\$1)" }, std::regex_constants::match_default);

With Boost, equivalent method/options can be used to achieve consistency in the results. Here, the default engines are used.

Regarding the MSVC and GCC differences, the documentation about that is scarce. It is clear that defining a literal backslash behavior is different between the two mentioned compiler. Note that a lot of regex libraries treat a literal backslash as a regex escape (same as in Boost, see reference above), and to define a literal replacement backslash, you need to double a literal backslash in the replacement pattern. The engine you used in GCC is ECMAScript.

It seems that how a backlash replacement pattern should be defined is left to each regex replace implementation. When you use it with GCC, a single literal \ (= "\\") is treated as a single literal replacement backslash. MSVC compiler decided to go with the majority of the regex engines and - which makes sense as you can use replacement backreferences as \1-\9 when using std::regex_constants::format_sed - requires a literal replacement backslash to be escaped and to replace with a single \, you need to use two literal backslashes, "\\\\" (or R"(\\)").