Mongo Spark connector and mongo 3.2, root user cannot read database

Question

I use the official mongo spark connector.

• my spark version is 2.0
• my mongo version is 3.2.x
• my spark mongo connector is 1.1.0

On my database i have one admin with root role, so he has all right.

i have created a config as follows :

     val readConfig = ReadConfig(Map("spark.mongodb.auth.uri" -> "mongodb://<userName>:<password>@<ip>:27017/admin",
"spark.mongodb.input.uri" -> "mongodb://<ip>:27017/MyDatabase.myCollection"))

but when i try to read some data i get an error "not authorized to execute command."

i don't understand why my root user is not authorized.

Answer 1

It's because "spark.mongodb.auth.uri" is not a configuration setting.

As the input uri doesn't have the authentication parameters the read is not authorised.

Try:

 val readConfig = ReadConfig(Map(
     "uri" -> "mongodb://<userName>:<password>@<ip>:27017/myDatabase.myCollection?authSource=admin"))

or:

 val readConfig = ReadConfig(Map(
     "uri" -> "mongodb://<userName>:<password>@<ip>:27017",  // uses the default db to auth against (admin)
     "database" -> "myDatabase",
     "collection" -> "myCollection"))

Answer 2

To avoid a full scan, you can do as below:

val rdd = MongoSpark.load(sc)

val aggregatedRdd = rdd.withPipeline(Seq(Document.parse("{ $match: { test : { $gt : 5 } } }")))
println(aggregatedRdd.count)
println(aggregatedRdd.first.toJson)