Apache Cassandra Failure on startup: FSWriteError (...) The operation is not permitted

Question

I've set up a small Cassandra Cluster with two nodes for testing purposes. On starting up Cassandra on the second node I'm getting this Exception:

INFO  [main] 2014-11-05 11:49:10,253 AutoSavingCache.java:123 - reading saved cache /var/lib/cassandra/saved_caches/system-local-7ad54392bcdd35a684174e047860b377-KeyCache-b.db
ERROR [main] 2014-11-05 11:49:10,855 CassandraDaemon.java:460 - Exception encountered during startup
org.apache.cassandra.io.FSWriteError: java.nio.file.FileSystemException: /var/lib/cassandra/data/system/compactions_in_progress-55080ab05d9c388690a4acb25fe1f77b/snapshots/1415184550829-compactions_in_progress/system-compactions_in_progress-ka-2-Summary.db -> /var/lib/cassandra/data/system/compactions_in_progress-55080ab05d9c388690a4acb25fe1f77b/system-compactions_in_progress-ka-2-Summary.db: Die Operation ist nicht erlaubt
    at org.apache.cassandra.io.util.FileUtils.createHardLink(FileUtils.java:98) ~[apache-cassandra-2.1.1.jar:2.1.1]
    at org.apache.cassandra.io.sstable.SSTableReader.createLinks(SSTableReader.java:1728) ~[apache-cassandra-2.1.1.jar:2.1.1]
    at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:2158) ~[apache-cassandra-2.1.1.jar:2.1.1]
    at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:2215) ~[apache-cassandra-2.1.1.jar:2.1.1]
    at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:2209) ~[apache-cassandra-2.1.1.jar:2.1.1]
    at org.apache.cassandra.db.ColumnFamilyStore$10.run(ColumnFamilyStore.java:2449) ~[apache-cassandra-2.1.1.jar:2.1.1]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_65]
    at org.apache.cassandra.db.ColumnFamilyStore.runWithCompactionsDisabled(ColumnFamilyStore.java:2515) ~[apache-cassandra-2.1.1.jar:2.1.1]
    at org.apache.cassandra.db.ColumnFamilyStore.truncateBlocking(ColumnFamilyStore.java:2462) ~[apache-cassandra-2.1.1.jar:2.1.1]
    at org.apache.cassandra.db.SystemKeyspace.discardCompactionsInProgress(SystemKeyspace.java:267) ~[apache-cassandra-2.1.1.jar:2.1.1]
    at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:234) [apache-cassandra-2.1.1.jar:2.1.1]
    at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:443) [apache-cassandra-2.1.1.jar:2.1.1]
    at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:532) [apache-cassandra-2.1.1.jar:2.1.1]
Caused by: java.nio.file.FileSystemException: /var/lib/cassandra/data/system/compactions_in_progress-55080ab05d9c388690a4acb25fe1f77b/snapshots/1415184550829-compactions_in_progress/system-compactions_in_progress-ka-2-Summary.db -> /var/lib/cassandra/data/system/compactions_in_progress-55080ab05d9c388690a4acb25fe1f77b/system-compactions_in_progress-ka-2-Summary.db: Die Operation ist nicht erlaubt
    at sun.nio.fs.UnixException.translateToIOException(UnixException.java:91) ~[na:1.7.0_65]
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[na:1.7.0_65]
    at sun.nio.fs.UnixFileSystemProvider.createLink(UnixFileSystemProvider.java:475) ~[na:1.7.0_65]
    at java.nio.file.Files.createLink(Files.java:1039) ~[na:1.7.0_65]
    at org.apache.cassandra.io.util.FileUtils.createHardLink(FileUtils.java:94) ~[apache-cassandra-2.1.1.jar:2.1.1]
    ... 12 common frames omitted

I know that by removing the keyspaces in /var/lib/cassandra/data/ the Exception isn't thrown anymore and Cassandra starts up. But there has to be a cleaner solution.

Answer 1

Most of the time this happens when you normally start cassandra with a special underprivileged user named for example cassandra. But the last time you started cassandra as root.

Now you have some files/folders that are owned by root and can't be accessed by the cassandra user.

To solve this problem you just need to run as root (if cassandra is that underprivileged user):
chown -R cassandra:cassandra /var/lib/cassandra/data/*

After doing so cassandra should start up normally.