Widening From Byte to Int (Marshmallow bug)

Question

Sorry in advance for the long question but everything should be straight forward and clear what's going on, thanks for taking a look. Please note that this is not actually code, just pseudo code to understand the application's implementation.

Issue

Byte's aren't widening to the true numerical value. Note: level = -1 represents a game that hasn't started. level == 24 represents the end of the game.

Class 1

private byte level = -1;
private byte stage = 0;

@NonNull private final Stages[][] stages = new Stages[25][13];

public byte getLevel () {
   return level;
}  

public void nextLevel () {
// expect: 0 
   level++;

// stages[  128][    0] (ArrayIndexOutOfBounds)
   stages[level][stage] = new Stage();
}

Class 2 extends Class 1

@NonNull private final byte[][] values = new byte[25][4];

public byte getScore (final byte level, final byte player) {
   //     values[  255][     2] (ArrayIndexOutOfBounds)
   return values[level][player];
}

// expecting: -1  >= 0 (false)
// runtime:   255 >= 0 (true)
if (Class1.getLevel() >= 0)
    getScore(Class1.getLevel(), 2);

Binary

8 bit (byte)

-1   == 1111 1111
-128 == 1000 0000
 127 == 0111 1111

32 bit (integer)

-1   == 1111 1111 1111 1111 1111 1111 1111 1111
 127 == 0000 0000 0000 0000 0000 0000 0111 1111
 128 == 0000 0000 0000 0000 0000 0000 1000 0000
 255 == 0000 0000 0000 0000 0000 0000 1111 1111

What Worked

Using the wrapper class

public Byte level = -1;

Question

I understand the issue is the number's binary representation is being used directly when it widens from byte to int. My number is literally going from 8 bit 1111 1111 to 32 bit 0000 0000 0000 0000 0000 0000 1111 1111. My question is why doesn't Java (or why is it not in this situation/environment) convert the number to the true numerical value when widening instead of just padding zeros to the raw binary representation.

This appears to only be happening to negative numbers, I assume because positive numbers have the same bit representation before and after widening.

Why isn't my number going from 8 bit 1111 1111 to 32 bit 1111 1111 1111 1111 1111 1111 1111 1111? Why is the postfix increment creating a value of 128..? Is there a better solution to this issue beside the one I'm currently stuck with.?

I prefer not to just use that solution without knowing the underlining problem because the issue can quietly (run without errors) break my applications algorithm; So it is with much appreciation if someone can please explain this issue to me.

Thanks, Jay

Current Working Environment

JDK 1.8.076

OS X El Capitan

Android Studio 2.2 Preview 2

buildToolsVersion '23.0.3'

classpath 'com.android.tools.build:gradle:2.2.0-alpha2'

Emulator Nexus 6P API 23 x86

Conclusion

I was able to narrow the issue to exclusively Android 23 (Marshmallow) devices. I have reported the bug to Google Inc. Thanks for everybody's help, I'll just give a warning to Android 23 (Marshmallow) user's as the bug doesn't appear in Android N, Android 22 and lower.

Answer 1

why not translate signed bytes to unsigned?

public static int signedByteToInt(byte b) {
    return b & 0xFF;
}

to make sure for you here is samples of representation in signed byte:

-3 - 11111101
-2 - 11111110
-1 - 11111111
0  - 00000000
1  - 00000001
2  - 00000010
3  - 00000011

and when you use byte as int it java will in anyway represent this byte as signed one so from 11111111 (-1) you will get 11111111 11111111 11111111 11111111 (-1) and I dont see here any problems.

just keep in memory that:

from -1 to -128 is from 11111111 to 10000000
and from 0 to 127 is from 00000000 to 01111111

and make proper conversion when you use it as int.

so below zero representations are like backward counter to middle

and by the way this is not only in java )

here:

if (Class1.getLevel() >= 0)

you compare byte with integer, try to make:

if (Class1.getLevel() >= (byte)0)

and feel happy :)

Answer 2

Jay, I saw your code and I checked the documentation for the Java language.

To me, the behavior you describe is as follows:

private byte level = -1;
private byte stage = 0;

You initialize bytes to -1 and 0. Later, you call nextLevel() and expect it to init stage[0][0], but instead, you receive a "out of bounds" exception. So, the first level is never reached.

public void nextLevel() {
// expect: 0
    level++;

// stages[  128][    0] (ArrayIndexOutOfBounds)
    stages[level][stage] = new Stage();
}

If the code you did not show in your sample does not create side effects producing the situation, the following code is a working sample, that should be sufficient to expose the issue:

package com.bytemagic;

import com.sun.istack.internal.NotNull;

/**
 * Created by thst on 01.06.2016.
 */
public class ByteMagic {

    private class Stage {
    }

    private byte level = -1;
    private byte stage = 0;

    @NotNull
    private final Stage[][] stages = new Stage[25][13];

    public byte getLevel() {
        return level;
    }

    public void nextLevel() {
    // expect: 0
        level++;

    // stages[  128][    0] (ArrayIndexOutOfBounds)
        stages[level][stage] = new Stage();
    }


    public static void main(String... args) {
        ByteMagic me = new ByteMagic();

        me.nextLevel();
        System.out.println(Integer.toHexString(me.getLevel()));
    }
}

I tried this code with JDK8u66, JDK8u77. This does not expose any issue. The byte level will properly be increased to 0 and the stages is initializing properly.

Would you please try to run that code on your machine and setup? Is this code exposing the issue?

After all: level cannot hold 128, it can hold -128, but that is not what you describe.

level++ will do all kinds of funny conversions. According to the docs, to calculate the increment, 1, level are converted to integer with proper widening if required (so 0xff -> 0xffffffff), then 1 is added, resulting in 0x0 (int). This is then narrowed back to 0x0 (byte) and written to level. (https://docs.oracle.com/javase/specs/jls/se8/html/jls-5.html#jls-5.1.2)

The array access is using a slightly different conversion rule. The array access requires the index to be of type integer. The conversion from byte to int uses unary numeric promotion rules, that can have interesting conversion effect when using unary minus, but for your case, with a byte value of 0, nothing spectacular or unexpected will happen.

(For reference: https://docs.oracle.com/javase/specs/jls/se8/html/jls-5.html#jls-5.6.1 and https://docs.oracle.com/javase/specs/jls/se8/html/jls-10.html#jls-10.4 and https://docs.oracle.com/javase/specs/jls/se8/html/jls-15.html#jls-15.10.3)

So, I cannot reproduce the failure with the code samples given, and my suspicion goes into the direction that this is either some side effect of code I cannot see or it is a bug in the JDK you use or in the environment (Dalvik VM).