Parsing a GLSL shader string to find the variable names in the Android NDK

Question

This one is a doozy. For the sake of proper explanation let me explain what I'm trying to do. I'll follow up with a code listing, and then explain the code aftwards.

The Goal

I'm trying to get the names of the variables in every GLSL shader file I have. Right now, I only have a single vertex shader, along with a fragment shader to complement it. The purpose of this is so I can dynamically bind values to the shaders without having to enter each and every single variable name.

Code

std::vector< const char* > GetShaderVariableNames( const Shader& shader )
    {
        Config::Log::info( "Getting shader variable names." );

        static const char* keyLookupTable[] =
        {
            "vec2", "vec3", "vec4",
            "mat2", "mat3", "mat4",
            "float", "int", "double"
        };

        std::vector< const char* >  keys;
        std::vector< std::string > lines;

        SplitIntoLines( &lines, std::string( shader.shaderSrc ) );

        for( int32_t iLines = 0; iLines < lines.size(); ++iLines )
        {
            const char* line = lines[ iLines ].c_str();

            int32_t index = 0;
            bool foundMatch = false;

            for( int32_t iKey = 0; iKey < sizeof( keyLookupTable ) / sizeof( char ); ++iKey )
            {
                if( strContains( lines[ iLines ], keyLookupTable[ iKey ] ) )
                {
                    index = iKey;
                    foundMatch = true;
                    break;
                }
            }

            if( foundMatch )
            {
                const int32_t pos = lines[ iLines ].find( keyLookupTable[ index ] );

                Config::Log::info( "Position found is %i", pos );

                const int32_t lineLen = strlen( line );

                char* var = new char[ lineLen - pos ];

                int32_t iLine = pos + strlen( keyLookupTable[ index ] );

                for( ; iLine < lineLen; ++iLine )
                {
                    var[ iLine ] = line[ iLine ];
                }

                Config::Log::info( "Shader Variable Found is: %s", var );

                keys.push_back( var );
            }
        }

        return keys;
    }

Taking the Red Pill

So, the idea is that there's a key look up table containing the most commonly used variable types. First off, the Shader received is a class which holds information about the data, such as its handle, its type (Fragment, Vertex, Texture, etc.), and of course its source. I'm parsing these all from shader files, and not strings.

What happens is there is a grand-daddy loop which iterates over each line parsed in the shader file. In each and every line, if there is a match in the key lookup table, the second loop iterating over keyLookupTable[] will break with an index value taking on the value of iKey (i.e., the index in the array, where the match is found). The loop then breaks.

If a match is found, the position in the line where the match is found (e.g. vec4 or mat3) is taken. From there, using the position stored in pos, we use pos to act as a basis for the length of the variable name, which is done by specifying the required amount of characters in a char array. The required amount is the length of the line, minus the position.

From there a third and final loop then iterates over the line, using a char* to reference it, taking the values in line and copying them to the allocated var character array.

Finally, the std::vector keys inserts var and continues on in the loop, repeating the process.

Notable Concerns

• I'm using the JNI to get the shader strings, as the shaders themselves are parsed via Java, and then sent through the JNI to C++.
• Unicode may be of a concern, as I've been getting outputs such as this: Shader Variable Found is: |uԯ|uԯ/
• The shader src is passed into a const char* from the JNI through env->GetStringUTFChars()

Conclusion

I'm sure there's a better way to do this, maybe using std::stringstream or something, but I'm not very familiar with it and would like this algorithm to work somehow or someway. However, if this is the "naive" way to do it, I'm open to suggestion.

The Question

What is the best way to achieve this to get the parsing to work?

Answer 1

Are you sure that you need to do this yourself? GLSL already does this parsing for you, and if you want a list of all the input variables you can get them via glGetActiveAttrib / glGetActiveUniform.

Just query the number of active attribs/uniforms of a linked shader, and then iterate over each index querying for the name of the input variable.

http://www.opengl.org/sdk/docs/man/xhtml/glGetActiveAttrib.xml

http://www.opengl.org/sdk/docs/man/xhtml/glGetActiveUniform.xml