std::stringstream to read int and strings, from a string

Question

I am programming in C++ and I'm not sure how to achieve the following:

I am copying a file stream to memory (because I was asked to, I'd prefer reading from stream), and and then trying to access its values to store them into strings and int variables.

This is to create an interpreter. The code I will try to interpret is (ie):

10 PRINT A
20 GOTO 10

This is just a quick example code. Now the values will be stored in a "map" structure at first and accessed later when everything will be "interpreted". The values to be stored are:

int lnum // line number

string cmd // command (PRINT and GOTO)

string exp // expression (A and 10 in this case but could hold expressions like (a*b)-c )

question is given the following code, how do I access those values and store them in memory? Also the exp string is of variable size (can be just a variable or an expression) so I am not sure how to read that and store it in the string.

code:

---

#include <iostream>
#include <fstream>
#include <string>
#include <cstdlib>
#include <cstring>
#include <map>
#include <sstream>

using namespace std;

 #include "main.hh"


int main () 
{
    int lenght;
    char *buffer;

// get file directory
    string dir;
    cout << "Please drag and drop here the file to interpret: ";
    getline (cin,dir);
    cout << "Thank you.\n";
    cout << "Please wait while your file is being interpreted.\n \n";

// Open File
    ifstream p_prog;
    p_prog.open (dir.c_str());

// Get file size
    p_prog.seekg (0, ios::end);
    lenght = p_prog.tellg();
    p_prog.seekg(0, ios::beg);

// Create buffer and copy stream to it
    buffer = new char[lenght];
    p_prog.read (buffer,lenght);
    p_prog.close();

// Define map<int, char>
    map<int, string> program;
    map<int, string>::iterator iter;


/***** Read File *****/
    int lnum; // line number
    string cmd; // store command (goto, let, etc...)
    string exp; // to be subst with expr. type inst.

//  this is what I had in mind but not sure how to use it properly
//  std::stringstream buffer;
//  buffer >> lnum >> cmd >> exp;

    program [lnum] = cmd; // store values in map




// free memory from buffer, out of scope
    delete[] buffer;
    return 0;
}

I hope this is clear.

Thank you for your help.

Valerio

Answer 1

You can use a std::stringstream to pull tokens, assuming that you already know the type.

For an interpreter, I'd highly recommend using an actual parser rather than writing your own. Boost's XPressive library or ANTLR work quite well. You can build your interpreter primitives using semantic actions as you parse the grammar or simply build an AST.

Another option would be Flex & Bison. Basically, these are all tools for parsing pre-defined grammars. You can build your own, but prepare for frustration. Recursively balancing parentheses or enforcing order of operations (divide before multiply, for example) isn't trivial.

The raw C++ parsing method follows:

#include <sstream>
#include <string>

// ... //

istringstream iss(buffer);
int a, b;
string c, d;

iss >> a;
iss >> b;
iss >> c;
iss >> d;

Answer 2

The way something like this can be done (especially the arithmetic expression part that you alluded to) is:

• Write some code that determines where a token ends and begins. For example 5 or + would be called a token. You might scan the text for these, or common separators such as whitespace.
• Write up the grammar of the language you're parsing. For example you might write:

    expression -> value
    expression -> expression + expression
    expression -> expression * expression
    expression -> function ( expression )
    expression -> ( expression )

Then based on this grammar you would write something that parses tokens of expressions into trees.

So you might have a tree that looks like this (pardon the ASCII art)

            +
          /   \
         5     *
              / \
             x   3

Where this represents the expression 5 + (x * 3). By having this in a tree structure it is really easy to evaluate expressions in your code: you can recursively descend the tree, performing the operations with the child nodes as arguments.

See the following Wikipedia articles:

• Parser
• Top-down parsing (your needs are probably simple enough to use this)
• Recursive-descent parser (a simple way to convert a grammar into code)

Or consult your local computer science department. :-)

There are also tools that will generate these parsers for you based on a grammar. You can do a search for "parser generator".