Update has no effect on scores (Prediction API)

Question

I'm experimenting with the language_id.txt dataset from the Google Prediction example. Right now I'm trying to update the model with the following method:

def update(label, data)   input = @prediction.trainedmodels.update.request_schema.new   input.label = label   input.csv_instance = [data]   result = @client.execute(     :api_method => @prediction.trainedmodels.update,     :parameters => {'id' => MODEL_ID},     :headers => {'Content-Type' => 'application/json'},     :body_object => input   )   assemble_json_body(result) end 

(This method is based on some Google sample code.)

My problem is that these updates have no effect. Here are the scores for This is a test sentence. regardless of how many updates I run:

{    "response":{       "kind":"prediction#output",       "id":"mymodel",       "selfLink":"https://www.googleapis.com/prediction/v1.5/trainedmodels/mymodel/predict",       "outputLabel":"English",       "outputMulti":[          {             "label":"English",             "score":0.420937          },          {             "label":"French",             "score":0.273789          },          {             "label":"Spanish",             "score":0.305274          }       ]    },    "status":"success" } 

Per the disclaimer at the bottom of "Creating a Sentiment Analysis Model", I have made sure to update at least 100 times before expecting any changes. First, I tried using a single sentence and updating it 1000 times. Second, I tried using ~150 unique sentences drawn from Simple Wikipedia and updated with each once. Each update was "successful":

{"response":{"kind":"prediction#training","id":"mymodel","selfLink":"https://www.googleapis.com/prediction/v1.5/trainedmodels/mymodel"},"status":"success"} 

but neither approach changed my results.

I've also tried using the APIs Explorer (Prediction, v1.5) and updating ~300 times that way. There's still no difference in my results. Those updates were also "successful".

200 OK { "kind": "prediction#training", "id": "mymodel", "selfLink": "https://www.googleapis.com/prediction/v1.5/trainedmodels/mymodel" } 

I am quite sure that the model is receiving these updates. get and analyze both show that the model has numberInstances": "2024". Oddly, though, list shows that the model has "numberInstances": "406".

At this point, I don't know what could be causing this issue.

Answer 1

2019 Update

Based on the comment from Jochem Schulenklopper that the API was shut down in April 2018.

Developers who choose to move to the Google Cloud Machine Learning Engine will have to recreate their existing Prediction API models.

Machine Learning API examples: https://github.com/GoogleCloudPlatform/cloudml-samples