App Engine ignores my source-code-updates and my app.yaml file

Question

I have my AppEngine app running fine with the development server on my desktop machine, but I cannot get the version on Google's server to pay attention to my source code updates.

Here is the most telling illustration. My app.yaml file started as:

application: xxxxxxxx
version: 1
runtime: python
api_version: 1

builtins:
- datastore_admin: on

handlers:
- url: /statix
  static_dir: statix

- url: /favicon\.ico
  static_files: favicon.ico
  upload: favicon\.ico

- url: .*
  script: main.py

The last line routes all non-static URLs to the main.py file. Let's call this version of main.py the "old" version.

The problems started after I made some routine changes to main.py (thereby creating the "new" version). The "new" version serves fine on my localhost machine, but its changes are not served over the Big Internet after I deploy to GAE. Instead, GAE servies the pages computed by "old" main.py. Sometimes quitting GAE Launcher and re-starting helped, but that hasn't worked in awhile now (hours). Even with a machine re-boot.

So, to experiment, I changed the last line of app.yaml to:

script: main2.py

I re-deployed to GAE, and when I hit my website, I get a 500 Error -- as expected -- because there is no main2.py file in the project.

So I rename the "new" main.py disk file to be main2.py, redeploy to GAE, and the 500 error goes away -- also as expected.

But here's where it gets weird. This just-uploaded main2.py file is the "new" main.py, simply renamed. BUT, the pages that are served by GAE are the pages computed by the "old" main.py!

I have cleared my browser cache. I have even fired up a dusty copy of Opera that has never seen the URLs served by this project, and it, too, is displaying the outdated page content that "old" main.py creates.

I have been using App Engine for a couple years. I've never had trouble like this before a week ago. I recently updated the Launcher to 1.7.3 (Mac OS X 10.6.8). At first I was thinking that my Launcher was somehow corrupted, but the app.yaml experiment suggests it's something in GAE itself. I've got a lot of projects in my GAE Launcher, and really don't want to do a clean install.

I bet I'm going to be embarrassed when I hear your thoughts, but I've run out of ideas.

ADDENDUM: User dragonx suggested that caching might be at the root of the problem, which, now that I think about it, did indeed begin after I configured a custom domain for the application, using the Google Domains service. But, here, according to curl -D, is the header on my application's home page:

HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Cache-Control: no-cache
Expires: Fri, 01 Jan 1990 00:00:00 GMT
Vary: Accept-Encoding
Date: Tue, 11 Dec 2012 02:29:43 GMT
Server: Google Frontend
Transfer-Encoding: chunked

It looks like caching is off. Still, I cannot find how to go back into the Google Domains service and turn off caching. :-(

ADDENDUM #2: I uploaded my files as a different app (i.e. another of the 10 that I'm allowed). They served the updates. However, when I made a change in the newly uploaded files, those changes did not appear. Hence, it appears that whatever the problem, it affects all of my apps under my user account.

Answer 1

Did you set HTTP cache headers? If so, you might be getting results from some intermediate cache, like GAE's edge cache.

Answer 2

Look at "version" and "admin logs" you may be updating the wrong version ie not the default https://appengine.google.com Also try without cookies (Cookies sometimes do weird things to the update)

appcfg.py update . --no_cookies