Arduino Reading from a constantly updating file

Question

I was wondering if you could help some of the high school students I am teaching by solving this problem that I have no idea how to fix. This is also helping them see what a wonderful resource Stackoverflow can be.

My students are trying to create a weather lamp with real time detection by using an arduino. A python program reads in what weather a zip code has using the Yahoo API and appends it to a file every 15 minutes or so.

At the same time, our Arduino is using Processing to access the file, push the numbers into the Serial Port and the arduino reads the serial port to turn on the correct lights to show the "weather" (sunny will turn on the yellow LEDs).

Our Processing and Arduino works fine (it reads from a file and shows the correct lights). It even works when the Processing and Arduino environments are running and you manually add numbers to the file.. Our Python file works fine as well by outputting the correct weather to the file.

The Problem... The two scripts can't run at the same time. If the Python is doing real world updates (checking the weather every 15 minutes), the Processing WILL not touch the file. The file will not be read until the Python script is COMPLETELY done and we start up the Processing environment (again). This defeats the purpose of real world updates and the point of this project, if it won't access the file and have the lights change over time.

On a related note, I know it would have been better to use the Wifi Shield for the Arduino, but we did not have the time nor resources to get it. It was 80 dollars and they have a week from start to finish to do this project.

Here is the code.

How the file looks

2, 3, 4, 3, 2

2 - sunny (turn on pin 2)... 3 - rainy (turn on pin 3)...

Arduino

 void setup() { 
     // initialize the digital pins as an output.
     pinMode(2, OUTPUT);
     pinMode(3, OUTPUT);
     pinMode(4, OUTPUT);
    // Turn the Serial Protocol ON
     Serial.begin(9600);
}

void loop() {
     byte byteRead;

     /* check if data has been sent from the computer: */
     if (Serial.available()) {
         /* read the most recent byte */
         byteRead = Serial.read();
         //You have to subtract '0' from the read Byte to convert from text to a number.
         byteRead=byteRead-'0';

         //Turn off all LEDs if the byte Read = 0
         if(byteRead==0){
             //Turn off all LEDS
             digitalWrite(2, LOW);
             digitalWrite(3, LOW);
             digitalWrite(4, LOW);

         }

         //Turn LED ON depending on the byte Read.
        if(byteRead>0){
             digitalWrite((byteRead), HIGH); // set the LED on
             delay(100);
         }
     }
 }

Processing

import processing.serial.*;
import java.io.*;
int mySwitch=0;
int counter=0;
String [] subtext;
Serial myPort;


void setup(){
     //Create a switch that will control the frequency of text file reads.
     //When mySwitch=1, the program is setup to read the text file.
     //This is turned off when mySwitch = 0
     mySwitch=1;

     //Open the serial port for communication with the Arduino
     //Make sure the COM port is correct
     myPort = new Serial(this, "COM3", 9600);
     myPort.bufferUntil('\n');
}

void draw() {
     if (mySwitch>0){
        /*The readData function can be found later in the code.
        This is the call to read a CSV file on the computer hard-drive. */
        readData("C:/Users/Lindsey/GWC Documents/Final Projects/lights.txt");

        /*The following switch prevents continuous reading of the text file, until
        we are ready to read the file again. */
        mySwitch=0;
     }
     /*Only send new data. This IF statement will allow new data to be sent to
     the arduino. */
     if(counter<subtext.length){
        /* Write the next number to the Serial port and send it to the Arduino 
        There will be a delay of half a second before the command is
        sent to turn the LED off : myPort.write('0'); */
        myPort.write(subtext[counter]);
         delay(500);
         myPort.write('0');
         delay(100);
         //Increment the counter so that the next number is sent to the arduino.
         counter++;
     } else{
         //If the text file has run out of numbers, then read the text file again in 5 seconds.
         delay(5000);
         mySwitch=1;
     }
} 


/* The following function will read from a CSV or TXT file */
void readData(String myFileName){

     File file=new File(myFileName);
     BufferedReader br=null;

     try{
        br=new BufferedReader(new FileReader(file));
        String text=null;

        /* keep reading each line until you get to the end of the file */
        while((text=br.readLine())!=null){
            /* Spilt each line up into bits and pieces using a comma as a separator */
            subtext = splitTokens(text,",");
        }
     }catch(FileNotFoundException e){
        e.printStackTrace();

     }catch(IOException e){
        e.printStackTrace();

     }finally{
        try {
            if (br != null){
                br.close();
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
     }
}

Python code

import pywapi
import time

# create a file
file = open("weatherData.txt", "w+")
file.close()

for i in range(0,10):
    weather = pywapi.get_weather_from_weather_com("90210")['current_conditions']['text']
    print(weather)

    if "rain" or "showers" in weather:
        weatherValue = "4"

    if "Sun" in weather:
        weatherValue = "2"
    if "thunder" in weather:
        weatherValue = "5"
    #elif "Cloud" in weather:
    if "Cloudy" in weather: 
        weatherValue = "3"
        print(weatherValue)

    # append the file with number
    # append with a comma after and space
    file = open("weatherData.txt","a")
    file.write(weatherValue + ", ")
    file.close()

    time.sleep(10)

Answer 1

This doesn't actually have to be a three part programming challenge - because you can use the PySerial module. This is a module I have used in the past to retrieve online data and pass it directly to the arduino via the serial port. To begin, you have to install the module as documented on the link I gave you. In the python program, you could alter your code to look like this:

import pywapi
import time
import serial #The PySerial module

# create a file
file = open("weatherData.txt", "w+")
file.close()

ser = serial.Serial("COM3", 9600) #Change COM3 to whichever COM port your arduino is in

for i in range(0,10):
    weather = pywapi.get_weather_from_weather_com("90210")['current_conditions']['text']
    print(weather)

    if "rain" or "showers" in weather:
        weatherValue = "4"

    if "Sun" in weather:
        weatherValue = "2"
    if "thunder" in weather:
        weatherValue = "5"
    #elif "Cloud" in weather:
    if "Cloudy" in weather: 
        weatherValue = "3"
        print(weatherValue)

    # append the file with number
    # append with a comma after and space
    file = open("weatherData.txt","a")
    file.write(weatherValue + ", ")
    file.close()

    #Sending the file via serial to arduino
    byte_signal = bytes([weatherValue])
    ser.write(byte_signal)

    time.sleep(10)

You don't even need to write it to a file, but if you plan on using the file in other ways the program should still create the same file.

Your arduino code could then look like this:

int weather_pin = 0;
void setup() {
  Serial.begin(9600);
  pinMode(2, OUTPUT);
  pinMode(3, OUTPUT);
  pinMode(4, OUTPUT);
}

void loop() {
  if(weather_pin>0){
    digitalWrite(weather_pin, HIGH);
  }
}

void serialEvent() {
  digitalWrite(2, LOW);
  digitalWrite(3, LOW);
  digitalWrite(4, LOW);
  weather_pin = Serial.read();
}

This should work! I have it working on my system, but every system is different so if it doesn't work feel free to drop me a comment. If you have any questions about the code, same deal. As a high school student myself who knows the beauty of stackoverflow I think it's absolutely awesome what you are doing for the kids. Best of luck in creating the weather lamp!