2D isometric - SFML - Right formulas, wrong coordinate range

Question

I don't work with tiles but cubes drawn with sf::Vertex. Each cubes have 6 sides with 4 points each.

So i just have to cubes[numCube].sides()[numSide].... to select a side.

I create cubes layer.cpp :

for(int J = 0; J < mapSize; J++)
    {
        for(int I = 0; I < mapSize; I++)
        {
            x = (J - I) * (cubeSize/2);
            y = (J + I) * (cubeSize/4);

            c = new cube(cubeSize, x, y, z, I, J);
            cs.push_back(*c);
        }
    }

In cube.cpp i create sides, then, in sides.cpp, i calcul each points' coordinates like this :

switch(typeSide)
{
    case 0://DOWN_SIDE
        light = 1;

        tmp_x = x + (size/2);
        tmp_y = y + (size/2);
        p0 = new point(tmp_x, tmp_y, tmp_z);

        tmp_x = x + size;
        tmp_y = y + (3 * (size/4));
        p1 = new point(tmp_x, tmp_y, tmp_z);

        tmp_x = x + (size/2);
        tmp_y = y + size;
        p2 = new point(tmp_x, tmp_y, tmp_z);

        tmp_x = x;
        tmp_y = y + (3 * (size/4));
        p3 = new point(tmp_x, tmp_y, tmp_z);
        break;

    case 1://BACK_LEFT_SIDE

//ETC. ....

Point.cpp :

/*
 * point.cpp
 *
 *  Created on: 21 nov. 2015
 *      Author: user
 */

#include "point.h"

point::point(float tx, float ty, float tz)
{
    coords* dummyVar = new coords(tx, ty, tz);
    coordinates = dummyVar;
}

std::vector<float> point::position()//Use : myPoint.getPosition[0] //get the x
{
    std::vector<float> dummyVar;

    dummyVar.push_back(coordinates->getX());
    dummyVar.push_back(coordinates->getY() - coordinates->getZ());

    return dummyVar;
}

void point::move(float tx, float ty, float tz)
{
    coordinates->setX(tx);
    coordinates->setY(ty);
    coordinates->setZ(tz);
}

My problem come from the function i use to detect click :

if (event.type == sf::Event::MouseMoved)
{
            currentSelectedCube = maps[currentMapID].getCubeIDAt(event.mouseMove.x, event.mouseMove.y, offsetLeft, offsetTop, enableOffset);
}

The function(don't bother with the comments) :

I try to get a cube's entry in my cube vector without 'for loop'. Why ? to use less CPU when i click.

int map::getCubeIDAt(float x, float y, int offsetLeft, int offsetTop, bool enableOffset)//WIP ! //USED FOR CLICK DETECTION ON CUBES
    {
    //----------------------------------------------------------------//
        int unsigned entry = -1;

        int I = 0;
        int J = 0;
    //----------------------------------------------------------------//

        if(currentLayerId() > -1)//If there is any layers
        {
            //IF CHECK IN MAP BOUDING BOX + ROTATION TO GOT DIAMOND SHAPE AREA(LAYER + OFFSETS)----------------------------------
            //{

                if(!enableOffset)//With offsets disabled
                {
                    I = (y * 2 - x) / cubeSize;
                    J = (y * 2 + x) / cubeSize;
                }
                else //With offsets enabled
                {
                    I = (((y-offsetTop)+(currentLayerId()*(cubeSize/2))) * 2 - (x-offsetLeft)) / cubeSize;
                    J = (((y-offsetTop)+(currentLayerId()*(cubeSize/2)))  * 2 + (x-offsetLeft)) / cubeSize;
                }

                entry = I + J * size;

                if (entry < 0 || entry >= layers()[currentLayerId()].cubes().size())
                {
                    entry = -1;
                }
                else//DEBUG - DISPLAYING VALUES FOR TEST
                {
                    std::cout << "Entry n°" << entry << " - ";
                    std::cout << "[" << I << "; " << J << "]" << std::endl;
                }
            //}
            //END IF CHECK IN MAP BOUDING BOX + ROTATION TO GOT DIAMOND SHAPE AREA(LAYER + OFFSETS)----------------------------------
        }

        return entry;
    }

The I-J and entryNumber are OK. i mean, for example, for the cube 0, i have I = 0; J = 0; etc ... This is working.

I don't understand why the coordinate range is like the red part(not accurate at 100%, i'm not a paint genius ha ha) in this picture :

But i should get that(2nd picture - the red part is where i click) :

But after few checks, the I-J and the entry i got are corresponding. This is so weird.

EDIT2: Offsets and layer number implemented. Problem left: wrong coordinates range.

Just in case, this is the 'function' handling events :

void GRAPHICS_HANDLER::listenEvents()
{
    while (window->pollEvent(event))
    {
        if (event.type == sf::Event::Closed)
        {
            window->close();
        }

        if(event.type == sf::Event::KeyPressed)
        {
            //DISPLAY/UNDISPLAY GRID -- DEBUG FUNCTION
            if(event.key.code == sf::Keyboard::Escape)
            {
                if(grid)
                    grid = false;
                else
                    grid = true;
            }

//-----------------------------------------------------------------------------------DEBUG---------------------------------------------------------------//
            if(event.key.code == sf::Keyboard::B)//ACTIVE BRUSHMODE -- NEED TO BLOCK IT WHEN ACCESS VIOLATION OF CUBES ARRAY(CRASH)
            {
                if(!brushMode)
                {
                    brushMode = true;
                    std::cout << "Brush mode enabled" << std::endl;
                }
                else
                {
                    brushMode = false;
                    std::cout << "Brush mode disabled" << std::endl;
                }
            }

            if(event.key.code == sf::Keyboard::L)//ADD_LAYER
            {
                addLayer(getCurrentMapID());
            }

            if(event.key.code == sf::Keyboard::M)//DELETE_LAYER
            {
                deleteLayer(currentMapID, maps[currentMapID].currentLayerId());
            }

            if(event.key.code == sf::Keyboard::S)//ADD_LAYER
            {
                std::cout << "Select a texture: ";
                std::cin >> currentSelectedTexture; std::cout << std::endl;
            }

            if(event.key.code == sf::Keyboard::Left)//Move in Layer
            {
                if(maps[currentMapID].currentLayerId() > 0)
                {
                    maps[currentMapID].setCurrentLayerID(maps[currentMapID].currentLayerId()-1);
                }
            }

            if(event.key.code == sf::Keyboard::Right)//Move in Layer
            {
                if(maps[currentMapID].currentLayerId() < maps[currentMapID].layers().size()-1)
                {
                    maps[currentMapID].setCurrentLayerID(maps[currentMapID].currentLayerId()+1);
                }
            }
//-----------------------------------------------------------------------------------DEBUG---------------------------------------------------------------//
        }

        if (event.type == sf::Event::MouseMoved)
        {
//--------------------------------------------------------------------------CURSOR-----------------------------------------------------------------------//
            currentSelectedCube = maps[currentMapID].getCubeIDAt(event.mouseMove.x, event.mouseMove.y, offsetLeft, offsetTop, enableOffset);
//--------------------------------------------------------------------------CURSOR-----------------------------------------------------------------------//
        }

        if (event.type == sf::Event::MouseButtonPressed)
        {
//--------------------------------------------------------------------------CURSOR-----------------------------------------------------------------------//
            currentSelectedCube = maps[currentMapID].getCubeIDAt(event.mouseButton.x, event.mouseButton.y, offsetLeft, offsetTop, enableOffset);
//--------------------------------------------------------------------------CURSOR-----------------------------------------------------------------------//
            if (event.mouseButton.button == sf::Mouse::Left)
            {
//--------------------------------------------------------------------------CUBE CLICK DETECTION--------------------------------------------------//
                if(maps.size() > 0 && maps[currentMapID].layers().size() > 0 && currentSelectedCube > -1)
                {
                    cubeClicked = true;
                }
            }

            if (event.mouseButton.button == sf::Mouse::Right)
            {
                if(maps.size() > 0 && maps[currentMapID].layers().size() > 0 && currentSelectedCube > -1)
                {
                    maps[currentMapID].layers()[maps[currentMapID].currentLayerId()].cubes()[currentSelectedCube].setTexture(1);
                }
            }
//--------------------------------------------------------------------------CUBE CLICK DETECTION--------------------------------------------------//
        }
    }
}

EDIT3: I updated my code to allow me to draw only the down side of the cube, so i can do this(the grass) :

The coordinate range(the red isometric square shown before in the screenshots) change a little when i put flat square(green). I don't know why, i prefer to precise it, just in case.

Answer 1

You need to store the "heigth" of each element from the tiles plane in order to distinguish which cube are you actually selecting (the closer to the observer):

Same screen coordinates, but different tiles.

It's not clear to me how you modeled your world, so I'll give you a partial algorithm to check what face of what cube is the one clicked. Please, adapt it to your actual code and to the classes you have written to make it work.

// I'll let you to add the offsets for the screen coordinates
I = (y * 2 - x) / cubeSize;
J = (y * 2 + x) / cubeSize;
// find out if it is a left or right triangle
if ( x < (J - I) * (cubeSize/2) ) {
    // left triangle
    for ( k = max_n_layer; k > -1; --k ) {
        // you create the cubes nesting the I loop in the J loop, so to get the index of a cube,
        // assuming that you have created all the cubes (even the invisible ones, like it seems from your code)
        index = (J+1+k)*mapsize + I+1+k;

        // I don't really get how you define the existence or not of a face, but I guess something like this:
        if ( index < map.layer[k].cubes.size() 
            &&  map.layer[k].cubes[index].sides[top_side] != 0 ) { 
        // the face selected is the top side of cube[index] of layer k
            // you have to return index and k to select the right face, or simply a pointer to that face
            // if this makes any sense with how you have designed your model
            return &map.layer[k].cubes[index].sides[top_side];
        }
        // now check for the side
        index = (J+k)*mapsize + I+1+k;
        if ( index < map.layer[k].cubes.size() 
            && map.layer[k].cubes[index].sides[right_side] != 0 ) { 

            return &map.layer[k].cubes[index].sides[right_side];
        }
        index = (J+k)*mapsize + I+k;
        if ( index < map.layer[k].cubes.size() 
            && map.layer[k].cubes[index].sides[left_side] != 0 ) { 

            return &map.layer[k].cubes[index].sides[left_side];
        }
    }
} else {
    // right triangle
    for ( k = max_n_layer; k > -1; --k ) {

        index = (J+1+k)*mapsize + I+1+k;

        if ( index < map.layer[k].cubes.size() 
            &&  map.layer[k].cubes[index].sides[top_side] != 0 ) { 
            return &map.layer[k].cubes[index].sides[top_side];
        }

        index = (J+1+k)*mapsize + I+k;
        if ( index < map.layer[k].cubes.size() 
            && map.layer[k].cubes[index].sides[left_side] != 0 ) { 

            return &map.layer[k].cubes[index].sides[left_side];
        }
        index = (J+k)*mapsize + I+k;
        if ( index < map.layer[k].cubes.size() 
            && map.layer[k].cubes[index].sides[right_side] != 0 ) { 

            return &map.layer[k].cubes[index].sides[right_side];
        }
    }
}    
// well, no match found. As I said is up to you to decide how to do in this case
return nullptr;

Edit

I suggest you to try another way.

Consider the screen as divided not by quadrangular tiles but by the triangles you already depicted. Every 2D tile of your model will be formed by two of those triangles and so all the sides of the cubes you want to draw. For every cube don't draw nor even create the back sides, those will never be drawn.

You can try to implement a sort of specialized z-buffer algorithm by storing for each one of the triangles you have to draw on the screen the index of the side which is closer to the observer. The coordinates of the vertex of all the triangles are calculated (once) with the code you already have.

            (I,J)              //For every node (I,J) you have a left and a right triangle
           .  *  .
(I+1,J) *  .  |  .  * (I,J+1)
              *
          (I+1,J+1)

You are creating your cubes layer by layer, I guess, each layer having a different heigth over the base plane. Create every side of the cube using the coordinates calculated earlier. For every face (only the 3 pointing to the observer) consider each one of its 2 triangles. You can easily determine if it is visible or not if you proceed in order, then you only have to update the ID stored in the corresponding triangle.

Once finished this fase, you'll have to draw each triangle once as you already have dropped the hidden ones. To determine the inverse transformation from screen coordinates to cell indexes, you only have to calculate which triangle is hitted and then look up which ID correspond to that. So transform back x,y to I,J (you already have those equations) and choose the left triangle if x < (J-I)/cubesize the right one otherwise.