Cannot create temp dir with proper permission: /mnt1/s3

Question

The following is the log dump of one of the container. I got an exception stating that a folder can't be created due to some permissions. I have troubleshooted various time but still it exist.

16/12/19 09:44:05 WARN ConfigurationUtils: Cannot create temp dir with proper permission: /mnt1/s3

java.nio.file.AccessDeniedException: /mnt1 at sun.nio.fs.UnixException.t here ranslateToIOException(UnixException.java:84) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:384) at java.nio.file.Files.createDirectory(Files.java:674) at java.nio.file.Files.createAndCheckIsDirectory(Files.java:781) at java.nio.file.Files.createDirectories(Files.java:767) at com.amazon.ws.emr.hadoop.fs.util.ConfigurationUtils.getTestedTempPaths(ConfigurationUtils.java:224) at com.amazon.ws.emr.hadoop.fs.s3n.S3NativeFileSystem.initialize(S3NativeFileSystem.java:449) at com.amazon.ws.emr.hadoop.fs.EmrFileSystem.initialize(EmrFileSystem.java:111) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2717) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:93) at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2751) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2733) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:377) at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295) at org.apache.hadoop.mapred.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:230) at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:201)

Answer 1

you should use the same instance type for your master and task/core nodes

see: https://forums.aws.amazon.com/thread.jspa?threadID=57967