Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

what's different between "mc cp --recursive" and "mc mirror --overwrite"

Tags:

minio

If we want to copy a bucket to another MiniO cluster, should we use "mc cp" or "mc mirror"? I have done some simple experiments and it seems that they are the same.

Thank~!

like image 314
Garlic Tseng Avatar asked Jan 02 '20 04:01

Garlic Tseng


People also ask

What is MC mirror?

The mc mirror command synchronizes content to MinIO deployment, similar to the rsync utility. mc mirror supports filesystems, MinIO deployments, and other S3-compatible hosts as the synchronization source. Note. mc mirror only synchronizes the current object without any version information or metadata.

How do you make a bucket in MinIO?

The mc mb command creates a new bucket or directory at the specified path. You can also use mc mb against the local filesystem to produce similar results to the mkdir -p commandline tool. The following command creates a new bucket mydata on the myminio MinIO deployment.


1 Answers

Short answer

Yes, mc cp --recursive SOURCE TARGET and mc mirror --overwrite SOURCE TARGET will have the same effect (to the best of my experience as of 2022-01).

  • mc cp allows for fine-tuned options for single files (but can bulk copy using --recursive)
  • mc mirror is focussed on bulk copying and can create buckets

Looking at the Minio client guide, there are several differences between the mc mirror and the mc cp commands, although the result of running them can be the same.

The answer to which one should you use depends on your requirements, and both options may be acceptable for you.

Details

The command signature differ: mc cp allows for multiple sources while mc mirror only allows for a single source.

In addition, the available flags are somewhat different (see below).

Flags mc cp offers not offered by mc mirror

  1. --rewind value: roll back object(s) to current version at specified time
  2. --version-id value, --vid value: select an object version to copy
  3. --attr: add custom metadata for the object (format: KeyName1=string;KeyName2=string)
  4. --continue, -c: create or resume copy session
  5. --tags: apply tags to the uploaded objects (eg. key=value&key2=value2, etc)
  6. --rewind value: roll back object(s) to current version at specified time
  7. (The --recursive, -r flag, but that's always true for mirror)

Flags offered by mc mirror not offered by mc clone:

Flags mc mirror offers not offered by mc cp

  1. --exclude value: exclude object(s) that match specified object name pattern
  2. --fake: perform a fake mirror operation
  3. --overwrite: overwrite object(s) on target if it differs from source
  4. --region value: specify region when creating new bucket(s) on target (default: "us-east-1")
  5. --watch, -w: watch and synchronize changes (This may be a big deal)

Consider using rclone as an alternative with additional flexibility. The Minio project is focussed on performance and being an excellent, simple S3 backend, not implementing every feature you could ask for (e.g., chunk size, throttling).

like image 162
Gordon Fogus Avatar answered Oct 29 '22 04:10

Gordon Fogus