Java swt - Get actual x,y coordinates from Image after scaling and zooming

Question

I have an image that has been scaled to fit. From the scaled image a user is selecting a rectangle.

I then re-draw based on this selection:

gc.drawImage(imageDisplayed, minX, minY, width, height, imageDisplayed.getBounds().x,  imageDisplayed.getBounds().y, imageDisplayed.getBounds().width, imageDisplayed.getBounds().height );

So now I want to be able to get the original co-ordinate from the scaled AND zoomed image. Is this correct?:

public Coordinate GetScaledXYCoordinate(int oldX, int oldY, int width, int height, int scaledWidth, int scaledHeight)
{       
    int newX = (int)(oldX * width)/scaledWidth;
    int newY = (int)(oldY * height)/scaledHeight;

    Coordinate retXY = new Coordinate(newX, newY);
    return retXY;
}


public Coordinate GetZoomedXYCoordinate(int oldX, int oldY, int startX, int endX, int startY, int endY,
        int width, int height,int scaledWidth, int scaledHeight)
{       
    // First get x,y after scaling
    Coordinate xy = GetScaledXYCoordinate(oldX, oldY, width, height, scaledWidth, scaledHeight);

    // Now get x.y after zooming
    int minX = Math.min(startX, endX);
    int minY = Math.min(startY, endY);

    int maxX = Math.max(startX, endX);
    int maxY = Math.max(startY, endY);

    int rectWidth = maxX - minX;
    int rectHeight = maxY - minY;
    return GetScaledXYCoordinate(xy.getX(), xy.getY(), width, height, scaledWidth, scaledHeight);
}

Note: I would like an algorithm that would work for many zooms, not just one zoom.

Update:

Ideally, I would like a function that takes a screen Point X,Y, and returns the original image X,Y. The function would still return the correct X,Y after scaling and zooming

Answer 1

The method selectionToOriginal should return a Rectangle with the position and dimension of the last zooming selection relative to the original image.

It receives:

• scaledDimensions: Point with the dimension of your scaled image, which is where the zooming selection are performed
• levels: List with the consecutive zooming Rectangle selections; in the first level you put the dimension of the original image

This test program show its use with an original image with dimension 800x600 and scaled dimension of 400x300. Two consecutive zooming selection are applied to it.

import java.util.ArrayList;
import java.util.List;

import org.eclipse.swt.graphics.Point;
import org.eclipse.swt.graphics.Rectangle;

public class ScaleTest {

    public static void main(String[] args) {

        Point scaledDimensions = new Point(400, 300);

        List<Rectangle> levels = new ArrayList<Rectangle>();

        // first level is the original image dimension
        levels.add(new Rectangle(0, 0, 800, 600));

        // other levels are the zooming selection inside the scaled image
        levels.add(new Rectangle(0, 0, 200, 150));
        levels.add(new Rectangle(200, 150, 200, 150));

        Rectangle selectionToOriginal = selectionToOriginal(scaledDimensions,
            levels);

        System.out.println(selectionToOriginal);
    }

    public static Rectangle selectionToOriginal(Point scaledDimensions,
        List<Rectangle> levels) {

        int numberOfLevels = levels.size();
        double scaledX = 0;
        double scaledY = 0;

        // we will work with the size of the last selection
        double scaledWidth = levels.get(numberOfLevels - 1).width;
        double scaledHeight = levels.get(numberOfLevels - 1).height;

        // start from the last selection to the first 
        for (int currentLevel = numberOfLevels - 1; currentLevel > 0; currentLevel--) {

            // get the width of the level N - 1
            double previousSelectionWidth = levels.get(currentLevel - 1).width;

            // convert the width of 1 unit in level N to its width in level N - 1
            double unitaryWidth = previousSelectionWidth / scaledDimensions.x;
            // convert the X position in level N in its X position in level N - 1
            scaledX = unitaryWidth * (levels.get(currentLevel).x + scaledX);
            // convert the width in level N in its width in level N - 1
            scaledWidth *= unitaryWidth;

            // get the height of the level N - 1
            double previousSelectionHeight = levels.get(currentLevel - 1).height;

            // convert the height of 1 unit in level N to its height in level N - 1
            double unitaryHeight = previousSelectionHeight / scaledDimensions.y;
            // convert the Y position in level N in its Y position in level N - 1
            scaledY = unitaryHeight * (levels.get(currentLevel).y + scaledY);
            // convert the height in level N in its height in level N - 1
            scaledHeight *= unitaryHeight;
        }

        return new Rectangle((int) scaledX, (int) scaledY, (int) scaledWidth,
            (int) scaledHeight);
    }

}

The program return a Rectangle with position (200, 150) and size (200, 150), the image show the situation:

Notes:

• in your code you used the class Coordinate which it seem equal to the SWT class Point which I used in my method

• the casts in the return instruction

  return new Rectangle((int) scaledX, (int) scaledY, (int) scaledWidth,
          (int) scaledHeight);
  

  will truncate the value of the doubles, consider using Math.round instead if you prefer to round the values

Answer 2

Here is a complete working example to zoom into an image using SWT which implements the idea behind Leon's answer. Using affine transformations is the default approach for drawing elements with individual coordinate systems in 2D graphics.

1. Use a Transform for drawing the picture at the right place and scale
2. Use the inverse of that Transform to get the image coordiantes of the selected zoom region.
3. Compute a new Transform to display the zoomed region.

The class below does the following:

• The Transform is stored in paintTransform.
• The screen coordiantes of the zoomed area are stored in zoomStart and zoomEnd
• The image coordinates of the selected area are computed in setVisibleImageAreaInScreenCoordinates from the dragged zoom rectangle.
• The new Transform is computed in setVisibleImageAreaInImageCoordinates
• Most of the rest can be considered boilerplate code.

Please note that the image is never replaced with a scaled version. It is drawn using the paintTransform. That means that the graphics context takes care of painting the image scaled. The actual painting code becomes as simple as

ev.gc.setTransform(paintTransform);
ev.gc.drawImage(img, 0, 0);

All the computation is done during the handling during the state transition triggered by the mouse events, i.e. the zoom() method called in the mouseUp() handler.

import java.io.InputStream;
import java.net.URL;

import org.eclipse.swt.events.MouseEvent;
import org.eclipse.swt.events.MouseListener;
import org.eclipse.swt.events.MouseMoveListener;
import org.eclipse.swt.events.PaintEvent;
import org.eclipse.swt.events.PaintListener;
import org.eclipse.swt.graphics.Color;
import org.eclipse.swt.graphics.GC;
import org.eclipse.swt.graphics.Image;
import org.eclipse.swt.graphics.ImageData;
import org.eclipse.swt.graphics.Point;
import org.eclipse.swt.graphics.RGB;
import org.eclipse.swt.graphics.Transform;
import org.eclipse.swt.widgets.Display;
import org.eclipse.swt.widgets.Shell;

public class Zoom implements PaintListener, MouseMoveListener, MouseListener {
    private static final int MOUSE_DOWN = 1;
    private static final int DRAGGING = 2;
    private static final int NOT_DRAGGING = 3;

    int dragState = NOT_DRAGGING;
    Point zoomStart;
    Point zoomEnd;

    ImageData imgData;
    Image img;
    Transform paintTransform;
    Shell shell;
    Color rectColor;

    public Zoom(ImageData image, Shell shell) {
        imgData = image;
        img = new Image(shell.getDisplay(), image);
        this.shell = shell;
        rectColor = new Color(shell.getDisplay(), new RGB(255, 255, 255));
    }

    void zoom() {
        int x0 = Math.min(zoomStart.x, zoomEnd.x);
        int x1 = Math.max(zoomStart.x, zoomEnd.x);
        int y0 = Math.min(zoomStart.y, zoomEnd.y);
        int y1 = Math.max(zoomStart.y, zoomEnd.y);

        setVisibleImageAreaInScreenCoordinates(x0, y0, x1, y1);
    }

    void setVisibleImageAreaInImageCoordinates(float x0, float y0,
            float x1, float y1) {
        Point sz = shell.getSize();

        double width = x1 - x0;
        double height = y1 - y0;

        double sx = (double) sz.x / (double) width;
        double sy = (double) sz.y / (double) height;

        float scale = (float) Math.min(sx, sy);

        // compute offset to center selected rectangle in available area
        double ox = 0.5 * (sz.x - scale * width);
        double oy = 0.5 * (sz.y - scale * height);

        paintTransform.identity();
        paintTransform.translate((float) ox, (float) oy);
        paintTransform.scale(scale, scale);
        paintTransform.translate(-x0, -y0);
    }

    void setVisibleImageAreaInScreenCoordinates(int x0, int y0,
            int x1, int y1) {
        Transform inv = invertPaintTransform();
        // points in screen coordinates
        // to be transformed to image coordinates
        // (top-left and bottom-right corner of selection)
        float[] points = { x0, y0, x1, y1 };

        // actually get image coordinates
        // (in-place operation on points array)
        inv.transform(points);
        inv.dispose();

        // extract image coordinates from array
        float ix0 = points[0];
        float iy0 = points[1];
        float ix1 = points[2];
        float iy1 = points[3];

        setVisibleImageAreaInImageCoordinates(ix0, iy0, ix1, iy1);
    }

    Transform invertPaintTransform() {
        // clone paintTransform
        float[] elems = new float[6];
        paintTransform.getElements(elems);
        Transform inv = new Transform(shell.getDisplay());
        inv.setElements(elems[0], elems[1], elems[2],
                        elems[3], elems[4], elems[5]);

        // invert clone
        inv.invert();
        return inv;
    }

    void fitImage() {
        Point sz = shell.getSize();

        double sx = (double) sz.x / (double) imgData.width;
        double sy = (double) sz.y / (double) imgData.height;

        float scale = (float) Math.min(sx, sy);

        paintTransform.identity();
        paintTransform.translate(sz.x * 0.5f, sz.y * 0.5f);
        paintTransform.scale(scale, scale);
        paintTransform.translate(-imgData.width*0.5f, -imgData.height*0.5f);
    }

    @Override
    public void paintControl(PaintEvent ev) {
        if (paintTransform == null) {
            paintTransform = new Transform(shell.getDisplay());
            fitImage();
        }

        ev.gc.setTransform(paintTransform);
        ev.gc.drawImage(img, 0, 0);

        if (dragState == DRAGGING) {
            drawZoomRect(ev.gc);
        }
    }

    void drawZoomRect(GC gc) {
        int x0 = Math.min(zoomStart.x, zoomEnd.x);
        int x1 = Math.max(zoomStart.x, zoomEnd.x);
        int y0 = Math.min(zoomStart.y, zoomEnd.y);
        int y1 = Math.max(zoomStart.y, zoomEnd.y);

        gc.setTransform(null);

        gc.setAlpha(0x80);
        gc.setForeground(rectColor);
        gc.fillRectangle(x0, y0, x1 - x0, y1 - y0);
    }

    public static void main(String[] args) throws Exception {
        URL url = new URL(
                "https://upload.wikimedia.org/wikipedia/commons/thumb/"  +
                "6/62/Billy_Zoom.jpg/800px-Billy_Zoom.jpg");
        InputStream input = url.openStream();
        ImageData img;
        try {
            img = new ImageData(input);
        } finally {
            input.close();
        }

        Display display = new Display();
        Shell shell = new Shell(display);
        shell.setSize(800, 600);

        Zoom zoom = new Zoom(img, shell);

        shell.open();
        shell.addPaintListener(zoom);
        shell.addMouseMoveListener(zoom);
        shell.addMouseListener(zoom);

        while (!shell.isDisposed()) {
            if (!display.readAndDispatch())
                display.sleep();
        }
        display.dispose();
    }

    @Override
    public void mouseDoubleClick(MouseEvent e) {
    }

    @Override
    public void mouseDown(MouseEvent e) {
        if (e.button != 1) {
            return;
        }
        zoomStart = new Point(e.x, e.y);
        dragState = MOUSE_DOWN;
    }

    @Override
    public void mouseUp(MouseEvent e) {
        if (e.button != 1) {
            return;
        }
        if (dragState == DRAGGING) {
            zoomEnd = new Point(e.x, e.y);
        }
        dragState = NOT_DRAGGING;
        zoom();
        shell.redraw();
    }

    @Override
    public void mouseMove(MouseEvent e) {
        if (dragState == NOT_DRAGGING) {
            return;
        }
        if (e.x == zoomStart.x && e.y == zoomStart.y) {
            dragState = MOUSE_DOWN;
        } else {
            dragState = DRAGGING;
            zoomEnd = new Point(e.x, e.y);
        }
        shell.redraw();
    }
}

When the window is resized, the transformation is currently not changed. That could be implemented in the same way as zooming: Compute previously visible image coordinates with old window size, compute new transformation with new window size.