What ist a RESTful-resource in the context of large data sets, i.E. weather data?

Question

So I am working on a webservice to access our weather forecast data (10000 locations, 40 parameters each, hourly values for the next 14 days = about 130 million values).

So I read all about RESTful services and its ideology.

So I understand that an URL is adressing a ressource.

But what is a ressource in my case?

The common use case is that you want to get the data for a couple of parameters over a timespan at one or more location. So clearly giving every value its own URL is not pratical and would result in hundreds of requests. I have the feeling that my specific problem doesn't excactly fit into the RESTful pattern.

Update: To clarify: There are two usage patterns of the service. 1. Raw data; rows and rows of data for several locations and parameters.

2. Interpreted data; the raw data calculated into symbols (Suns & clouds, for example) and other parameters.

There is not one 'forecast'. Different clients have different needs for data.

The reason I think this doesn't fit into the REST-pattern is, that while I can actually have a 'forecast' ressource, I still have to submit a lot of request parameters. So a simple GET-request on a ressource doesn't work, I end up POSTing data all over the place.

Answer 1

So I am working on a webservice to access our weather forecast data (10000 locations, 40 parameters each, hourly values for the next 14 days = about 130 million values). ... But what is a ressource in my case?

That depends on the details of your problem domain. Simply having a large amount of data is not a good reason to avoid REST. There are smart ways and dumb ways to model and expose that data.

As you rightly see, your main goal at this point should be to understand what exactly a resource is. Knowing only enough about weather forecasting to follow the Weather Channel, I won't be much help here. It's for domain experts like yourself to make that call.

If you were to explain in a little more detail the major domain concepts you're working with, it might make it a little easier to give specific advice.

For example, one resource might be Forecast. When weatherpeople talk about Forecasts, what words keep coming up? When you think about breaking a forecast down into smaller elements, what words do you use to describe the pieces?

Do this process recursively, and you'll probably be able to make a list of important terms. Don't forget that these terms can describe things or actions. Think about what these terms really mean, what data you can use to model them, how they can be aggregated.

At this point you'll have the makings of something you can start building a RESTful system around - but not before.

Don't forget that a RESTful system is not a data dump wrapped in HTTP - it's a hypertext-driven system.

Also don't forget that media types are the point of contact between your server and its clients. A media type is only limited by your imagination and can model datasets of any size if you're clever about it. It can contain XML, JSON, YAML, binary elements such as a Bloom Filter, or whatever works for the problem.