Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

MySQL BLOB image data gradually losing?

In a mysql MyISAM table, I have a column type mediumblob and storing captured image as blob data. I got some interesting and problematic images. Some of the images are gradually losing data.

Field          type  
--------------------------
image         mediumblob

my.ini max allowed packet size set max_allowed_packet = 8M

image1image2image3

this is the problem

When the C# application fetches the data from the server, this kind of images losing data of random sizes every time. I got 10-12 bad images like this in 100000+ image data.

What could be the reason of this kind of behavior? Anyone has any idea/solution how to fix/avoid this problem.

Update 1:
Reading bytes form PictureBox

MemoryStream ms = new MemoryStream();
byte[] ret = null;

try
{
     picturebox.Image.Save(ms, System.Drawing.Imaging.ImageFormat.Jpeg);
     byte[] Data = new byte[ms.Length];
     ms.Read(Data, 0, (int)ms.Length);
     ret = byteData;
     ms.Close();
 }         
 

Saving the bytes array into database as medium blob data. When retrieving the data from database I am casting the reader data:

byte[] Data = (byte[])reader["Image"];
like image 387
Nikson Kanti Paul Avatar asked Jun 13 '12 06:06

Nikson Kanti Paul


1 Answers

First of all, as Sarke mentioned, storing files content in DB isn't the best idea (file meta data is a whole different story.

Why?

  1. Performance: in majority of cases OS file cache will outperform anything built into DBMS.
  2. Disaster recovery: odds of loosing all/most files on failure are way higher than with file system and recovery is far more difficult
  3. Scaling: if you outgrow capacity of a single server adding application level sharding is trivial and with no performance penalty. Multiserver DB setups are more "painful"
  4. Multiple solutions available/ease of migration: There are plenty hardware and software solutions for large file collections storage and migrating between them is far more simple than migrating between DBMS

I store close to 2 million images that are stored in a simple folder structure: /xx/yy/filename, where filename = md5 of file (+ optional number should a hash collision happen), xx = first 2 characters of md5, yy = 3rd and 4th character of md5. It works great and I shouldn't get any FS related slowdowns for a long while (2 orders of magnitude at least).

Getting back to your question there are 3 options

  1. The files are never saved correctly to the DB. It might be problem in the app that is uploading the photos or the image is too big. Your max_allowed_packet restricts image size to ~8 MB, mediub_blob can store maximum of 16 MB. To rule this one out increase max_allowed_packet to 32 MB and test. You'll need to make sure no image exceeds this size at any point and make sure the app does it's job right when uploading photos. If you can find an image that was uploaded and displayed fine (from DB!) and later it didn't then this is not the cause.
  2. The files get damaged during update - if anything updates the photos in any way then even if original file is fine the updated one might not be - it might for example exceed the size limits from point 1.
  3. (the least likely one) If the file is stored and updated without damaging it then it is getting damaged while being stored -> with no reported MySQL bugs on this (and this wouldn't go unnoticed) I'd look on server hardware.
like image 136
c2h5oh Avatar answered Sep 29 '22 20:09

c2h5oh