Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

scylla sstableloader giving error on copying sstables from cassandra to scylla?

I am using Cassandra 3.0.15v and want to move my data to scylla db 3.3.0v using the sstableloader utility provided by scylla and i have tried different approaches but i am not able to do it.

Table schema is,

CREATE TABLE events.test (
    "Id1" text,
    "Id2" text,
    "event" set<text>,
    PRIMARY KEY ("Id1", "Id2")
) 
WITH CLUSTERING ORDER BY ("Id2" ASC)
    AND bloom_filter_fp_chance = 0.01
    AND caching = {'keys': 'ALL', 'rows_per_partition': 'ALL'}
    AND comment = ''
    AND compaction = {'class': 'SizeTieredCompactionStrategy'}
    AND compression = {'sstable_compression': 'org.apache.cassandra.io.compress.LZ4Compressor'}
    AND dclocal_read_repair_chance = 0.1
    AND default_time_to_live = 0
    AND gc_grace_seconds = 864000
    AND max_index_interval = 2048
    AND memtable_flush_period_in_ms = 0
    AND min_index_interval = 128
    AND read_repair_chance = 0.0
    AND speculative_retry = '99.0PERCENTILE';

The steps that i followed were,

  1. took the snapshot of the table.
  2. copied it on the scylla server.
  3. Run command: sstableloader --nodes serverIP keySpace/tableName

On running above command the response it get is:

com.datastax.driver.core.exceptions.SyntaxError: line 1:75  : missing elements...

I am not able to understand what am i doing wrong here. So any help would be highly appreciated.

Just found these logs using journalctl _COMM=scylla

cql_server - exception while processing connection: std::system_error (error system:32, sendmsg: Broken pipe)
Apr 22 14:54:02 ip-1-0-4-100 scylla[1371]:  [shard 0] cql_server - exception while processing connection: std::system_error (error system:32, sendmsg: Broken pipe)
Apr 22 14:54:02 ip-1-0-4-100 scylla[1371]:  [shard 2] cql_server - exception while processing connection: std::system_error (error system:32, sendmsg: Broken pipe)
Apr 22 15:35:33 ip-1-0-4-100 scylla[1371]:  [shard 3] cql_server - exception while processing connection: std::system_error (error system:32, sendmsg: Broken pipe)
lines 3484-3523/3523 (END)
like image 967
Yash Tandon Avatar asked Jan 25 '23 03:01

Yash Tandon


1 Answers

There are some minor differences in the table attributes between Cassandra and Scylla. That could be the cause for the error. It requires Minor changes in your Schema

Have you tried following the steps from this Cassandra to Scylla Migration doc? https://docs.scylladb.com/operating-scylla/procedures/cassandra_to_scylla_migration_process/

Specifically note the schema diff at the bottom of the doc. And also the limitations and known issues.

Another good source is this blog about migration strategies (if you are willing to consider other methods than sstableloader), there's also a link there to a webinar on the topic. https://www.scylladb.com/2019/04/02/spark-file-transfer-and-more-strategies-for-migrating-data-to-and-from-a-cassandra-or-scylla-cluster/

Good luck!

like image 136
TomerSan Avatar answered May 16 '23 08:05

TomerSan