What is the best way to test if a CharacterSet contains a Character in Swift 4?

Question

I'm looking for a way, in Swift 4, to test if a Character is a member of an arbitrary CharacterSet. I have this Scanner class that will be used for some lightweight parsing. One of the functions in the class is to skip any characters, at the current position, that belong to a certain set of possible characters.

class MyScanner {
  let str: String
  var idx: String.Index
  init(_ string: String) {
    str = string
    idx = str.startIndex
  }
  var remains: String { return String(str[idx..<str.endIndex])}

  func skip(charactersIn characters: CharacterSet) {
    while idx < str.endIndex && characters.contains(str[idx])) {
      idx = source.index(idx, offsetBy: 1)
    }
  }
}

let scanner = MyScanner("fizz   buzz fizz")
scanner.skip(charactersIn: CharacterSet.alphanumerics)
scanner.skip(charactersIn: CharacterSet.whitespaces)
print("what remains: \"\(scanner.remains)\"")

I would like to implement the skip(charactersIn:) function so that the above code would print buzz fizz.

The tricky part is characters.contains(str[idx])) in the while - .contains() requires a Unicode.Scalar, and I'm at a loss trying to figure out the next step.

I know I could pass in a String to the skip function, but I'd like to find a way to make it work with a CharacterSet, because of all the convenient static members (alphanumerics, whitespaces, etc.).

How does one test a CharacterSet if it contains a Character?

Answer 1

I know that you wanted to use CharacterSet rather than String, but CharacterSet does not (yet, at least) support characters that are composed of more than one Unicode.Scalar. See the "family" character (👩‍👩‍👧‍👦) or the international flag characters (e.g. "🇯🇵" or "🇯🇲") that Apple demonstrated in the string discussion in WWDC 2017 video What's New in Swift. The multiple skin tone emoji also manifest this behavior (e.g. 👩🏻 vs 👩🏽).

As a result, I'd be wary of using CharacterSet (which is a "set of Unicode character values for use in search operations"). Or, if you want to provide this method for the sake of convenience, be aware that it will not work correctly with characters represented by multiple unicode scalars.

So, you might offer a scanner that provides both CharacterSet and String renditions of the skip method:

class MyScanner {
    let string: String
    var index: String.Index

    init(_ string: String) {
        self.string = string
        index = string.startIndex
    }

    var remains: String { return String(string[index...]) }

    /// Skip characters in a string
    ///
    /// This rendition is safe to use with strings that have characters
    /// represented by more than one unicode scalar.
    ///
    /// - Parameter skipString: A string with all of the characters to skip.

    func skip(charactersIn skipString: String) {
        while index < string.endIndex, skipString.contains(string[index]) {
            index = string.index(index, offsetBy: 1)
        }
    }

    /// Skip characters in character set
    ///
    /// Note, character sets cannot (yet) include characters that are represented by
    /// more than one unicode scalar (e.g. 👩‍👩‍👧‍👦 or 🇯🇵 or 👰🏻). If you want to test
    /// for these multi-unicode characters, you have to use the `String` rendition of
    /// this method.
    ///
    /// This will simply stop scanning if it encounters a multi-unicode character in
    /// the string being scanned (because it knows the `CharacterSet` can only represent
    /// single-unicode characters) and you want to avoid false positives (e.g., mistaking
    /// the Jamaican flag, 🇯🇲, for the Japanese flag, 🇯🇵).
    ///
    /// - Parameter characterSet: The character set to check for membership.

    func skip(charactersIn characterSet: CharacterSet) {
        while index < string.endIndex,
            string[index].unicodeScalars.count == 1,
            let character = string[index].unicodeScalars.first,
            characterSet.contains(character) {
                index = string.index(index, offsetBy: 1)
        }
    }

}

Thus, your simple example will still work:

let scanner = MyScanner("fizz   buzz fizz")
scanner.skip(charactersIn: CharacterSet.alphanumerics)
scanner.skip(charactersIn: CharacterSet.whitespaces)
print(scanner.remains)  // "buzz fizz"

But use the String rendition if the characters you want to skip might include multiple unicode scalars:

let family = "👩\u{200D}👩\u{200D}👧\u{200D}👦"  // 👩‍👩‍👧‍👦
let boy = "👦"

let charactersToSkip = family + boy

let string = boy + family + "foobar"  // 👦👩‍👩‍👧‍👦foobar

let scanner = MyScanner(string)
scanner.skip(charactersIn: charactersToSkip)
print(scanner.remains)                // foobar

---

As Michael Waterfall noted in the comments below, CharacterSet has a bug and doesn’t even handle 32-bit Unicode.Scalar values correctly, meaning that it doesn’t even handle single scalar characters properly if the value exceeds 0xffff (including emoji, amongst others). The String rendition, above, handles these correctly, though.

Answer 2

Not sure if it's the most efficient way but you can create a new CharSet and check if they are sub/super-sets (Set comparison is rather quick)

let newSet = CharacterSet(charactersIn: "a")
// let newSet = CharacterSet(charactersIn: "\(character)")
print(newSet.isSubset(of: CharacterSet.decimalDigits)) // false
print(newSet.isSubset(of: CharacterSet.alphanumerics)) // true