Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Creating UI Image from Texture2D

I working on a live stream App that receives JPEG image as arrays of bytes and displays it on the screen with UI.Image. It works fine but I am making optimization and have few questions. Currently, the code I have below converts arrays of bytes to Texture2D then creates a Sprite from the Texture2D then assign that Sprite to UI.Iamge to display on the screen.

Texture2D camTexture;
Image screenDisplay;
public byte[] JPEG_VIDEO_STREAM;
bool updateScreen = false;

//Initializing

JPEG_VIDEO_STREAM = new byte[20000];
camTexture = new Texture2D(Screen.width, Screen.height, TextureFormat.RGB24, false);

//Main Code that runs in the Update function

if(updateScreen){
camTexture.LoadImage(JPEG_VIDEO_STREAM);
Sprite tempSprite = Sprite.Create(camTexture, new Rect(0, 0, camTexture.width, camTexture.height), Vector2.zero, 0);
screenDisplay.sprite = tempSprite;
updateScreen = false;
}

The code above currently perform 3 steps just to display image to screen.

byte array -> Texture2D -> Sprite -> UI.Image.

but I want it to look like byte array -> Texture2D-> UI.Image.

I want to write Texture2D directly to UI.Image without creating new Sprite because I believe that Sprite.Create(camTexture, new Rect(0, 0, camTexture.width, camTexture.height), Vector2.zero, 0); allocates new memory each time Sprite.Create called. I looked at the Unity Documentation and couldn't find any other way to do this.

My questions are:

  1. How can I assign camTexture(Texture2D) to the screen screenDisplay(UI.Image) without converting camTexture(Texture2D) to Sprite first?

  2. Does Sprite.Create allocate new memory when called?

  3. If there is a solution to this, is that solution better than what I currently have in terms of performance and memory management?

Note: I have no plans on using OnGUI to draw Texture2D. I want to do this with the new Unity UI. Thanks.

Edit: With Joe's answer of RawImage, the final code looks like this:

RawImage screenDisplay;
if(updateScreen){
camTexture.LoadImage(JPEG_VIDEO_STREAM);
screenDisplay.texture = camTexture;
updateScreen = false;
}

No more Sprite needed.

like image 327
Programmer Avatar asked Oct 19 '22 13:10

Programmer


1 Answers

I think that by specifically using a RawImage rather than Image, one can do this.

I use RawImage extensively, because, we have to "display PNGs" and it's easier.

Consider the very handy trick:

just start with a trivial gray PNG which you have imported .. and then modify that .. rather than try to build from scratch?

An interesting curiosity I found is: normally to mirror an image, you just simply scale of x or y to -1. Unless it's been fixed, Unity has a problem where this won't work for RawImage.

    // currently in Unity, the ONLY way to mirror a RAW image is by fooling with
    // the uvRect. changing the scale is completely broken.

    if ( shouldWeMirror )
      rawImage.uvRect = new Rect(1,0,-1,1); // means mirror
    else
      rawImage.uvRect = new Rect(0,0,1,1);  // means no flip

Another interesting factor. For this reason, many Unity projects still use (even 2017) the superlative 2dToolkit. It instantly solves issues such as this.

like image 190
Fattie Avatar answered Oct 21 '22 06:10

Fattie