R plot background map from Geotiff with ggplot2

Question

With the R base plot, I can plot any geotiff with the following command:

library("raster")
plot(raster("geo.tiff"))

For example, downloading this data, I would do the follwing:

setwd("C:/download") # same folder as the ZIP-File
map <- raster("smr25musterdaten/SMR_25/SMR_25KOMB_508dpi_LZW/SMR25_LV03_KOMB_Mosaic.tif")

How do you Plot GeoTif Files in ggplot2?

EDIT:

1: I've replaced the greyscale map from the sample files with a coloured map to ilustrate the problem of the missing colortable.

2: With the help of Pascals answer, I was able to adapt and improve this solution and make it more dynamic to the input tif. I will post the answer below.