TeX: string parsing and whitespace stripping

Question

In T_EX, how is it possible to split a string such as \mana{X1 2W/UB R /G} into parts and to feed them to another macro (in this case, to replace the macro invocation by something like \m{X}\m{12}\m{W/U}\m{B}\m{R/G}), grouping by very simple rules, namely: a) consecutive digits form a group, b) a slash creates a group of two nearby characters, c) spaces are to be stripped completely?

I tried the substr package but it wasn’t too helpful, only allowing one to find certain substrings. Hand-written loops such as

\def\lcpass#1.{}
\def\lcloop#1#2.{%
    \ifempty{#2}%
        #1%
        \let\continue=\lcpass%
    \else%
        \let\continue=\lcloop%
    \fi%
    \continue#2.}

\def\lastchar#1{\lcloop#1.} % returns the last character of a string

fail to work when the string ends in whitespace, neither was I successful with \futurelet.

In general, how does one approach the task of parsing strings in T_EX? For example, the texmate package allows one to write things like |1 e4 e5 Nf3 Nc6| and automatically draws corresponding chess positions. How does it do it? What can I read about looping over characters in a string and other T_EX hacks like this?

Answer 1

\def\m#1{\par$m$({\tt #1})}% Any macros
\def\removespaces{\catcode`\ =9 }% Ignore all spaces`

\let\manaNext\relax % aux def
\let\manaLastChar\relax % aux def
\newtoks\manaToks % aux toks
\newif\ifDigitProcessing

\def\mana#{\afterassignment \manaA \let\next= }% always next = {
\def\manaA{\bgroup \removespaces \let\manaNext\manaB \manaNext}% algorithm init: ignore spaces
\def\manaB{\futurelet\next\manaC}% algorithm start
\def\manaC{\ifx\next\egroup \def\nnext{\manaFlush\aftergroup\manaNext}\else\let\nnext\manaD\fi\nnext}% check for \egroup
\def\manaD{\ifx\next/\let\nnext\manaSlash\else \ifcat\next 1\let\nnext\manaDigit \else \let\nnext\manaE \fi\fi \nnext}% cases
\def\manaE#1{\manaFlush\DigitProcessingfalse\let\manaLastChar\next\manaNext}% Letters A-Z and a-z case
\def\manaFlush{\ifx\manaLastChar\relax\else\m{\manaLastChar}\fi\let\manaLastChar\relax
               \ifDigitProcessing\expandafter\m\expandafter{\the\manaToks}\fi\manaToks{}}% transform to \m{...}
\def\manaSlash#1#2{\m{\manaLastChar/#2}\let\manaLastChar\relax\manaNext}%#1=/, #2=next letter
\def\manaDigit#1{\ifDigitProcessing\else\manaFlush\fi
       \manaToks=\expandafter{\the\manaToks#1}\DigitProcessingtrue\manaNext}% 0-9 case

\hrule\medskip
\mana{X1 2W/UB R /G}
\medskip\hrule\medskip

\mana{X1 2W/UB s/SS 14 1 R /G XZ}
\medskip\hrule\medskip