is `addingPercentEncoding` broken in Xcode 9?

Question

in Swift 3.x with Xcode 9 beta 2, using addingPercentEncoding gives unexpected results. CharacterSet.urlPathAllowed always contains ":", so by definition of addingPercentEncoding, it should never escape it. Yet, using this code:

// always true
print(CharacterSet.urlPathAllowed.contains(":"))
let myString = "info:hello world"
let escapedString = myString.addingPercentEncoding(withAllowedCharacters: .urlPathAllowed)!
print(escapedString)

I get those results:

cases where I get an undesirable behavior

• Xcode 9 beta 2, iOS 9.3
• Xcode 9 beta 2, iOS 11.0

  true
  info%3Ahello%20world

cases where I get the expected behavior

• Xcode 9 beta 2, iOS 10.3.1
• Xcode 8.3.3, any iOS

  true
  info:hello%20world

Is there any workaround to get a working implementation of addingPercentEncoding that will correctly respect the given allowedCharacters?

Answer 1

Apparently there is some undocumented magic done by addingPercentEncoding when the CharacterSet used as reference is an underlying NSCharacterSet class.

So to workaround this magic, you need to make your CharacterSet a pure Swift object. To do so, I'll create a copy (thanks Martin R!), so that the evil magic is gone:

let myString = "info:hello world"
let csCopy = CharacterSet(bitmapRepresentation: CharacterSet.urlPathAllowed.bitmapRepresentation)
let escapedString = myString.addingPercentEncoding(withAllowedCharacters: csCopy)!
//always "info:hello%20world"
print(escapedString)

As an extension:

extension String {
    func safeAddingPercentEncoding(withAllowedCharacters allowedCharacters: CharacterSet) -> String? {
        // using a copy to workaround magic: https://stackoverflow.com/q/44754996/1033581
        let allowedCharacters = CharacterSet(bitmapRepresentation: allowedCharacters.bitmapRepresentation)
        return addingPercentEncoding(withAllowedCharacters: allowedCharacters)
    }
}

Answer 2

The reason that it is now percent escaping the : character is that .urlPathAllowed now strictly observes RFC 3986, which says in section 3.3, “Paths”:

In addition, a URI reference (Section 4.1) may be a relative-path reference, in which case the first path segment cannot contain a colon (":") character.

Thus, the : is permitted in relative paths (which is what we're dealing with here), but simply not in the first component.

Consider:

let string = "foo:bar/baz:qux"
print(string.addingPercentEncoding(withAllowedCharacters: .urlPathAllowed)!)

That will, in conformance with RFC 3986, percent encode the : in the first component, but allow it unencoded in subsequent components:

foo%3Abar/baz:qux

This character set is not percent encoding solely on the basis of what characters are in the set, but is actually applying RFC 3986's relative path logic. But as Cœur said, if you need, you can bypass this logic by building your own character set with the same allowed characters as .urlPathAllowed, and that new character set will not apply this RFC 3986 logic.