Git format-patch patches can't use LFS across different repositories

Question

We are currently migrating from CVS to a git repository. One team however already used cvs2git about a year ago for their repository because their new feature needs a lot of moving files with history which CVS is historically not very good at. The repositories both should then use GIT-LFS for our bigger checked in sources (pre-built external libraries, for example)

The idea was to create the same starting point for merging all their changes with history onto our repository once we migrate.

So I already have an exact replica of the starting point; the idea was to create now all the patch files with git format-patch. Now most of the patches are working fine, the problem is just with patches that use git lfs.

an example:

diff --git a/Foo/bar/baz.zip b/Foo/bar/baz.zip
new file mode 100644
index 00000000000..6fce3f4bd05
--- /dev/null
+++ b/Foo/bar/baz.zip
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c90b657621e07fa40476186d94e2f6e06f055b49294b83ee976f73dfac120d86
+size 116056
-- 
2.13.2.windows.1

when I try to apply that patch, the following happens:

$ git am ../patches/mypatch.patch
Applying: #someTfsId: myCommit
Downloading Foo/bar/baz.zip (113.34 KB)
Error downloading object: Foo/bar/baz.zip (c90b657): Smudge error: Error opening media file.: open D:\repositories\myrepo\.git\lfs\objects\c9\0b\c90b657621e07fa40476186d94e2f6e06f055b49294b83ee976f73dfac120d86: The system cannot find the file specified.

Errors logged to D:\repositories\myrepo\.git\lfs\objects\logs\20171006T110837.3254847.log
Use `git lfs logs last` to view the log.
error: external filter 'git-lfs filter-process' failed
fatal: Foo/bar/baz.zip: smudge filter lfs failed

user@machine MINGW64 /d/repositories/myrepo (develop|AM 1/1)
$ git am --abort

my next idea was to use the original repository and get all lfs objects with GIT_TRACE=1 git lfs fetch --all afterwards I tried to copy that one object (for testing purposes) which resulted in

user@machine MINGW64 /d/repositories/myrepo (develop)
$ git am ../patches/mypatch.patch
error: Foo/bar/baz.zip: already exists in working directory
Applying: #someTfsId: myCommit
Patch failed at 0001 #someTfsId: myCommit
The copy of the patch that failed is found in: .git/rebase-apply/patch
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

So I suppose the question is: is it possible for me to add the LFS objects to the remote server, or do I have to "un-lfs" the repository that is used to create the .patch files, so the binary diffs are stored in the .patch files?

We really want to keep the history and at the moment the amount of patches would be about 1600 commits, which would be hard to do by hand.

If I am missing some obvious way I am happy for any constructive input

Answer 1

I know this question is quite old but I was just faced with this scenario today.

This is the way that I have solved the issue, perhaps there is a better way but this worked for me.

The basic steps are:

1. commit (but don't push) your changes
2. lfs migrate the files that otherwise would have been put into lfs
3. undo the commit (assuming you want your files in LFS)
4. revert the changes to .gitattribute for the files you want in LFS
5. recommit the files

Note: any uncommitted work will be lost in step 2 so be careful

Lets say I have a file called opensrc/src/make-4.3.tar.gz in the repository and I am at the top level of the repository. I want to create a patch that includes that file and the other files that were changed along with it but my tracked files are:

[root@localhost]# git lfs track
Listing tracked patterns
    *.gz (.gitattributes)
    *.sbr  (.gitattributes)
    *.tgz  (.gitattributes)
    *.exe  (.gitattributes)
Listing excluded patterns

Go ahead and add and commit your files that will be part of the patch. You can find out which files are going to be in LFS by performing:

[root@localhost]# git lfs status
On branch master
Objects to be pushed to origin/master:

    opensrc/src/make-4.3.tar.gz (e05fdde47c5f7ca45cb697e973894ff4f5d79e13b750ed57d7b66d8defc78e19)

Objects to be committed:


Objects not staged for commit:

So before I create the patch I add this file (and any others) to the exclude list:

[root@localhost]# git lfs migrate export --include opensrc/src/make-4.3.tar.gz
migrate: override changes in your working copy?  All uncommitted changes will be lost! [y/N] y
migrate: changes in your working copy will be overridden ...
migrate: Fetching remote refs: ..., done.                                                                                                                                                                          
migrate: Sorting commits: ..., done.                                                                                                                                                                               
migrate: Rewriting commits: 100% (1/1), done.                                                                                                                                                                      
  master   526f3a2488679349653bea096fc311c5f42c38dd -> 3734d32f6226c542f470e252aedfcafd5e0950cf
migrate: Updating refs: ..., done.                                                                                                                                                                                 
migrate: checkout: ..., done.                                                                                                                                                                                      
prune: 341 local object(s), 341 retained, done.                                                                                                                                                                    

e05fdde47c5f7ca45cb697e973894ff4f5d79e13b750ed57d7b66d8defc78e19 (2.3 MB)                                                                                                                                         
e05fdde47c5f7ca45cb697e973894ff4f5d79e13b750ed57d7b66d8defc78e19 (2.3 MB),    done.
prune: Deleting objects: 100% (1/1), done.          

Now checking lfs status again:

[root@localhost]# git lfs status
On branch master
Objects to be pushed to origin/master:

Objects to be committed:

Objects not staged for commit:

and now create the patch:

[root@localhost]# git format-patch HEAD -1
0001-Update-make.patch
[root@localhost]# **ls -l 0001-Update-make.patch** 
-rw-r--r--. 1 root root 2986800 Oct 21 18:44 0001-Update-make.patch

Now you can revert the LFS change and recommit the original so that your file ends up in LFS:

[root@localhost]# git reset HEAD^
Unstaged changes after reset:
M   .gitattributes
M   opensrc/Makefile
M   opensrc/config.common.mk
M   opensrc/rootproject/Makefile

[root@localhost]# git diff .gitattributes
diff --git a/.gitattributes b/.gitattributes
index 893a1726c..f94922c84 100644
--- a/.gitattributes
+++ b/.gitattributes
@@ -12,3 +12,4 @@
 *.sbr filter=lfs diff=lfs merge=lfs -text
 *.tgz filter=lfs diff=lfs merge=lfs -text
 *.exe filter=lfs diff=lfs merge=lfs -text
+opensrc/src/make-4.3.tar.gz !text !filter !merge !diff

[root@localhost]# git checkout .gitattributes
Updated 1 path from the index

[root@localhost]# git add .

[root@localhost]# git commit -v -s

NOTE: your patch will also exclude the file(s) from LFS so you may have to fix that by resetting the commit, editing .gitattributes and then recommitting