Search queries in microservice architecture

Question

Assuming I have a microservice architecture where I have a Book microservice holding all book details and a Author microservice holding all author details, e.g.

Book Service
GET /     get all books
GET /id   get book details, including author_ids
POST /    create new book
etc.

Author Service
GET /     get all authors
GET /id   get author details, including book_ids
POST /    create new author
etc.

The real services are much more enterprise grade, I just choose Books and Authors with their own data storage option as a easy understandable example.

Assuming, there will be millions of calls to the services and I have to uphold specific availability and throughput of systems. How can I efficiently search for authors where name starts with 'A' and all books written by the authors where book title starts with 'B'?

I see following options, which all are not perfect:

1. I create a search endpoint in Author service, fetch all authors matching the search criteria, follow each book_id and filter for the books. -> This requires a lot of calls on the Book service.

2. Same as 1. but I create a search endpoint in Book service, fetch all books matching the search criteria, follow each author_id and filter for the authors. -> This requires a lot of calls on the Author service. In worst case, same load as in 1.

3. I create a new microservice Search. Search will have its own database which is optimized for searches. Search will return me the books and authors and can give me the search result with one call. -> This requires Search to frequently sync with Book and Author service.

4. I merge Book and Author into one service which defeats the purpose of mircroservices?

Maybe someone with more microservice experience can help me out how to best architect this.

Answer 1

Well, You are correct that there will be a lot of calls if you choose to go with option 1 or 2. If you choose 3, You can use a search technology like ElasticSearch (which can be used both as a search technology as well as data storage as documents) OR you can use SOLR technology backed by a database of your choice. Whenever you add a Book to the Books Microservice, raise an event which carries data about books and your search service would denormalize this event data into JSON documents and store it in Elasticsearch. Similarly, listen to events from Author Microservice and denormalize them into the database.

In order to avoid a lot of calls, what you can also do is maintain all data about Books in Author Microservice (along with data about Author) and all data about Authors in Books Microservice (along with data about Books). There will be a lot of redundancy (and there will be problems like data synchronization) but at least now you won't have to do any cross-service calls.

Answer 2

You should stop worrying about normalizing your microservice so much. This is one of the harder bits to microservices, is to stop thinking about them like they are fully normalized database tables.

I'd store your author names along with your books. You can have extended author attributes in the author service, and you can still have an author ID to link the two, but keep enough denormalized author information to perform your search in that book service.

Then the call to search is simple... the potential downside, that author information is out-of-date on the books, is probably ok... if someone changes an author name, you will have to have a process to update it from the author service, but that's a FAR more scalable process than looking up authors for each call.