Are there limits on string length in Arduino?

Question

I've been trying for hours to put together a simple JSON object string on an arduino to send to a Raspberry Pi running node.

I cannot seem to successfully build the string. I have tried building the string all in one go:

"{" + string1 + "," + string2 + "," + string3 + "}" etc...

I've also tried using a the String.replace function. Each time I end up with a bit of my string, or none at all. The below code shows what's happening:

String msg = "{ \"message\" : \"statusUpdate\", ";
String active = " \"active\" : TOKEN, ";
String intakeTemp = " \"intakeTemp\" : TOKEN, ";
String intakeHumid = " \"intakeHumid\" : TOKEN, ";
String exhaustTemp = " \"exhaustTemp\" : TOKEN, ";
String exhaustHumid = " \"exhaustHumid\" : TOKEN, ";
String targetHumid = " \"targetHumid\" : TOKEN, ";
String completed = " \"taskCompleted\" : TOKEN }";


if(isActive)
  active.replace("TOKEN","true");
else
  active.replace("TOKEN","false");

intakeTemp.replace("TOKEN",floatToString(intakeTemperature,0));
intakeHumid.replace("TOKEN",floatToString(intakeHumidity,0));
exhaustTemp.replace("TOKEN",floatToString(exhaustTemperature,0));
exhaustHumid.replace("TOKEN",floatToString(exhaustHumidity,0));
targetHumid.replace("TOKEN",floatToString(targetHumidity,0));

if(taskFinished)
  taskCompleted.replace("TOKEN","true");
else
  taskCompleted.replace("TOKEN","false");



  String body = msg;
  Serial.println(body);
  body += active;
  Serial.println(body);
  body += intakeTemp;
  Serial.println(body);
  body += intakeHumid;
  Serial.println(body);
  body += exhaustTemp;
  Serial.println(body);
  body += exhaustHumid;
  Serial.println(body);
  body += targetHumid;
  Serial.println(body);
  body += taskCompleted;
  Serial.println(body);

You can see from the last bit of code above that as the string is being built, i'm spitting it out to the serial monitor. However, here is my serial output:

{ "message" : "statusUpdate", 
{ "message" : "statusUpdate",  "active" : false, 
{ "message" : "statusUpdate",  "active" : false,  "intakeTemp" : 0.0, 
{ "message" : "statusUpdate",  "active" : false,  "intakeTemp" : 0.0,  "intakeHumid" : 0.0, 
{ "message" : "statusUpdate",  "active" : false,  "intakeTemp" : 0.0,  "intakeHumid" : 0.0,  "exhaustTemp" : 0.0, 
{ "message" : "statusUpdate",  "active" : false,  "intakeTemp" : 0.0,  "intakeHumid" : 0.0,  "exhaustTemp" : 0.0, 
{ "message" : "statusUpdate",  "active" : false,  "intakeTemp" : 0.0,  "intakeHumid" : 0.0,  "exhaustTemp" : 0.0, 
{ "message" : "statusUpdate",  "active" : false,  "intakeTemp" : 0.0,  "intakeHumid" : 0.0,  "exhaustTemp" : 0.0, 

Is there a limit to the string length? I haven't found any mention of such limits in the docs. There's not much else to the sketch except the standard Ethernet library and the code to send it via an HTTP request (from the sample project).

Any idea what might be happening?

EDIT: Ok, I've shortened my string like so:

String msg = "{ \"m\" : \"status\", ";
String active = " \"a\" : TOKEN, ";
String intakeTemp = " \"iT\" : TOKEN, ";
String intakeHumid = " \"iH\" : TOKEN, ";
String exhaustTemp = " \"eT\" : TOKEN, ";
String exhaustHumid = " \"eH\" : TOKEN, ";
String targetHumid = " \"tH\" : TOKEN, ";
String dryerJustFinished = " \"f\" : TOKEN }";

and sure enough, it's started to work:

{ "m" : "status", 
{ "m" : "status",  "a" : false, 
{ "m" : "status",  "a" : false,  "iT" : 0.0, 
{ "m" : "status",  "a" : false,  "iT" : 0.0,  "iH" : 0.0, 
{ "m" : "status",  "a" : false,  "iT" : 0.0,  "iH" : 0.0,  "eT" : 0.0, 
{ "m" : "status",  "a" : false,  "iT" : 0.0,  "iH" : 0.0,  "eT" : 0.0,  "eH" : 0.0, 
{ "m" : "status",  "a" : false,  "iT" : 0.0,  "iH" : 0.0,  "eT" : 0.0,  "eH" : 0.0,  "tH" : 0.0, 
{ "m" : "status",  "a" : false,  "iT" : 0.0,  "iH" : 0.0,  "eT" : 0.0,  "eH" : 0.0,  "tH" : 0.0,  "f" : false }

Which means there is a limitation. Is this a memory limitation?

By the way, the hardware is an Arduino Uno R3

Answer 1

The arduino has very limited memory -- about 2K for your data (32K for your program in flash). Under the covers the String class is doing a realloc call for every string concatenation. This could lead to fragmented memory where there isn't a single contiguous block of memory available for the string.

I would suggest you stop using the String class and just allocate a large buffer at the beginning and append to it, but you will need to have an idea of how big it will be before you allocate it.

If you like to live dangerously, you could look at the source code for the String class and see if you can force it to pre-allocate a large buffer, but I see no documented way to do that.

Answer 2

Atmel processor has fairly limited memory management so its easy to end up with fragmented memory. Remember the runtime stack and heap also limited.

Static string can be put into PROGMEM also

There is also a freememory function on the arduino.cc that will show you how much free memory you have.

Answer 3

I had similar difficulties trying to build JSON using the Arduino String class. Eventually I gave up because I realized I was trying to force large system techniques onto a tiny embedded system. Now I use preallocated char arrays and C functions like sprintf for string manipulation, and I encode data in the most compact way that I can still use from my application.

You might find this link interesting (I didn't write it):

Five things I never use in Arduino projects....

1. The String class

On the face if it, the String class in the Arduino library makes string handling easier. It provides a host of functions to do things that you can’t easily do if you represent strings as pointers to char arrays, as is usual in C. So what’s the problem?

The problem is that String operations allocate memory dynamically and in ways that are hard to predict when the inputs to the program are variable, combined with the fact that Arduinos have a very limited amount of RAM (2K on the Arduino Uno). Dynamic memory allocation typically causes memory fragmentation. This means that your program may work correctly for some inputs or a short while, but crashes with other inputs or after a longer time, due to memory exhaustion. See http://critical.eschertech.com/2010/07/30/dynamic-memory-allocation-in-critical-embedded-systems/ for more about why dynamic memory allocation is a bad idea in embedded software implemented in C/C++.

When might I use String? When writing a quick-and-dirty sketch to try something out, as long it doesn’t have to hold together for more than a few minutes!