Add a file inside a folder to Github using JGit or EGit - directory1\myfile.txt

Question

*Added final working program to the bottom of this question *

I was able to add a file to GitHub using org.eclipse.egit.github.core java library (and code sample is referred from here: https://gist.github.com/Detelca/2337731 )

But I am not able to give a path like "folder1\myfile.txt" OR "folder1\folder2\myfile.txt"

I am trying to find a simple example to add a file under a folder and was not really able to find it.

Any help on this with an example?

Below is the code i am using: (configuration details like username, reponame are in the method addFileToGH() )

import java.io.IOException;
import java.util.ArrayList;
import java.util.Calendar;
import java.util.Collection;
import java.util.Date;
import java.util.List;

import org.eclipse.egit.github.core.Blob;
import org.eclipse.egit.github.core.Commit;
import org.eclipse.egit.github.core.CommitUser;
import org.eclipse.egit.github.core.Reference;
import org.eclipse.egit.github.core.Repository;
import org.eclipse.egit.github.core.RepositoryCommit;
import org.eclipse.egit.github.core.Tree;
import org.eclipse.egit.github.core.TreeEntry;
import org.eclipse.egit.github.core.TypedResource;
import org.eclipse.egit.github.core.User;
import org.eclipse.egit.github.core.client.GitHubClient;
import org.eclipse.egit.github.core.service.CommitService;
import org.eclipse.egit.github.core.service.DataService;
import org.eclipse.egit.github.core.service.RepositoryService;
import org.eclipse.egit.github.core.service.UserService;

public class GHFileWriter {

    public static void main(String[] args) {
        try {
            new GHFileWriter().addFileToGH("myfile.txt", "some file content");
            //new GHFileWriter().addFolderToGH("folder1");
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
    }


    public boolean addFileToGH(String fileName, String fileContent) throws IOException{
        String userName = "myusername";
        String password = "mypassword";
        String repoName = "myrepository"; 
        String ownerName = "myusername";
        String email = "[email protected]";

        //Basic authentication
        GitHubClient client = new GitHubClient();
            client.setCredentials(userName, password);

        // create needed services
        RepositoryService repositoryService = new RepositoryService();
        CommitService commitService = new CommitService(client);
        DataService dataService = new DataService(client);

     // get some sha's from current state in git
        Repository repository =  repositoryService.getRepository(ownerName, repoName );
        String baseCommitSha = repositoryService.getBranches(repository).get(0).getCommit().getSha();
        RepositoryCommit baseCommit = commitService.getCommit(repository, baseCommitSha);
        String treeSha = baseCommit.getSha();

        // create new blob with data
        Blob blob = new Blob();
            blob.setContent( fileContent ).setEncoding(Blob.ENCODING_UTF8); //"[\"" + System.currentTimeMillis() + "\"]").setEncoding(Blob.ENCODING_UTF8);
        String blob_sha = dataService.createBlob(repository, blob);
        Tree baseTree = dataService.getTree(repository, treeSha);
Collection<TreeEntry> entries = new ArrayList<TreeEntry>();

    // create new tree entry
    TreeEntry treeEntry = new TreeEntry();
        treeEntry.setPath( fileName );
        treeEntry.setMode(TreeEntry.MODE_BLOB);
        treeEntry.setType(TreeEntry.TYPE_BLOB);
        treeEntry.setSha(blob_sha);
        treeEntry.setSize(blob.getContent().length());

   /* //Code for adding folder, not working
TreeEntry folderEntry = new TreeEntry();
        folderEntry.setPath( "folder1" );
        folderEntry.setMode(TreeEntry.MODE_DIRECTORY);
        folderEntry.setType(TreeEntry.TYPE_TREE);
        folderEntry.setSha(blob_sha); //one of the possible issues
        //folderEntry.setSize(blob.getContent().length());

        entries.add(folderEntry);

*/              
entries.add(treeEntry);

        Tree newTree = dataService.createTree(repository, entries, baseTree.getSha()); //Issue is popped-up here, error is thrown here

     // create commit
        Commit commit = new Commit();
            commit.setMessage("Test commit at " + new Date(System.currentTimeMillis()).toLocaleString());
            commit.setTree(newTree);



            //Due to an error with github api we have to to all this
            //TODO: Make this better (another function)
            UserService userService = new UserService(client);
            User user = userService.getUser();

            CommitUser author = new CommitUser();
            Calendar now = Calendar.getInstance();

                author.setName( userName );
                author.setEmail(email);
            //author.setEmail(userService.getEmails().get(0));
                author.setDate(now.getTime());
                commit.setAuthor(author);
                commit.setCommitter(author);


        List<Commit> listOfCommits = new ArrayList<Commit>();
            listOfCommits.add(new Commit().setSha(baseCommitSha));
        // listOfCommits.containsAll(base_commit.getParents());
            commit.setParents(listOfCommits);
        // commit.setSha(base_commit.getSha());
        Commit newCommit = dataService.createCommit(repository, commit);

     // create resource
        TypedResource commitResource = new TypedResource();
        commitResource.setSha(newCommit.getSha());
        commitResource.setType(TypedResource.TYPE_COMMIT);
        commitResource.setUrl(newCommit.getUrl());

        //System.out.println("Committed file URL: "+ newCommit.getUrl());
     // get master reference and update it
        Reference reference = dataService.getReference(repository, "heads/master");
            reference.setObject(commitResource);
            dataService.editReference(repository, reference, true);

            if( newCommit.getUrl() != null || !newCommit.getUrl().equalsIgnoreCase("") ){
                return true;
            }
        //push.setCommits(commits);

        return false;
    }
}

Below is the error when i execute the above code.

org.eclipse.egit.github.core.client.RequestException: tree.sha a3b93733a6dfd221c24e94d2ce52c25307910d73 is not a valid tree (422) at org.eclipse.egit.github.core.client.GitHubClient.createException(GitHubClient.java:552) at org.eclipse.egit.github.core.client.GitHubClient.sendJson(GitHubClient.java:643) at org.eclipse.egit.github.core.client.GitHubClient.post(GitHubClient.java:757) at org.eclipse.egit.github.core.service.DataService.createTree(DataService.java:203) at com.apps.ui5.accelerator.writers.GHFileWriter.addFileToGH(GHFileWriter.java:87) at com.apps.ui5.accelerator.writers.GHFileWriter.main(GHFileWriter.java:30)

Edit: The error causing code is commented. The commented code starts with "//Code for adding folder, not working"

Possible issue is with this line of code: folderEntry.setSha(blob_sha);

The error is thrown at this line: Tree newTree = dataService.createTree(repository, entries, baseTree.getSha());

---

Edit(VonC response): Hi VonC, thanks a lot for your response. JGit looks promising but i see below issues.

1. I am running this as a service in a cloud instance where i dont know if i can rely on filerepo which is refererred in the cookbook example. If this works in cloud, i can consider that as a solution.
2. The example is also adding a simple file but not a folder/directory.
3. And I need to push changes to GitHub using a java program, not git commands. How can i do that?

Can you refer any example which is creating a folder? I tried below code for adding a folder but did not work!

import java.io.File;
import java.io.IOException;
import java.io.PrintWriter;

import org.dstadler.jgit.helper.CookbookHelper;
import org.eclipse.jgit.api.Git;
import org.eclipse.jgit.api.errors.GitAPIException;
import org.eclipse.jgit.api.errors.NoFilepatternException;
import org.eclipse.jgit.lib.Repository;


public class GHFolderWriter {

    public static void main(String[] args) throws IOException, NoFilepatternException, GitAPIException {
        // prepare a new test-repository
        try (Repository repository = CookbookHelper.createNewRepository()) {
            try (Git git = new Git(repository)) {
                //create folder
                File folder = new File(repository.getDirectory().getParent(), "folder1");
                if(!folder.createNewFile()) {
                    throw new IOException("Could not create folder " + folder);
                }

                // create the file
                File myfile = new File(folder, "testfile1.txt");
                if(!myfile.createNewFile()) {
                    throw new IOException("Could not create file " + myfile);
                }

                // Stage all files in the repo including new files
                git.add().addFilepattern(".").call();

                // and then commit the changes.
                git.commit()
                        .setMessage("Commit all changes including additions")
                        .call();

                try(PrintWriter writer = new PrintWriter(myfile)) {
                    writer.append("Hello, world!");
                }

                // Stage all changed files, omitting new files, and commit with one command
                git.commit()
                        .setAll(true)
                        .setMessage("Commit changes to all files")
                        .call();


                System.out.println("Committed all changes to repository at " + repository.getDirectory());
            }
        }

    }

}

It is throwing this error:

Exception in thread "main" java.io.IOException: Could not open file. Reason : The system cannot find the path specified (path C:\Users\ramgood\AppData\Local\Temp\TestGitRepository441020326444846897\folder1\testfile1.txt, working dir C:\Users\ramgood\workspace-sapui5-1.32\accelerate) at java.io.WinNTFileSystem.createFileExclusively(Native Method) at java.io.File.createNewFile(File.java:1016) at com.apps.ui5.accelerator.writers.GHFolderWriter.main(GHFolderWriter.java:37)

Edit(3): I moved to jgit as you suggested and below is my new program. I am able to 1. clone remote repo with my local repo 2. add a folder and file to local repo and commit locally but 3. Failing to push to github.

import java.io.File;
import java.io.IOException;
import java.io.PrintWriter;

import org.eclipse.jgit.api.Git;
import org.eclipse.jgit.api.errors.GitAPIException;
import org.eclipse.jgit.lib.Repository;
import org.eclipse.jgit.storage.file.FileRepositoryBuilder;
import org.eclipse.jgit.transport.UsernamePasswordCredentialsProvider;



/**
 * Note: This snippet is not done and likely does not show anything useful yet
 *
 * @author dominik.stadler at gmx.at
 */
public class PushToRemoteRepository {

    private static final String REMOTE_URL = "https://github.com/debsap/testrepo.git";

    public static void main(String[] args) throws IOException, GitAPIException {
        // prepare a new folder for the cloned repository
        File localPath = File.createTempFile("REStGitRepository", "");
        if(!localPath.delete()) {
            throw new IOException("Could not delete temporary file " + localPath);
        }

        // then clone
        System.out.println("Cloning from " + REMOTE_URL + " to " + localPath);
        try (Git git = Git.cloneRepository()
                .setURI(REMOTE_URL)
                .setCredentialsProvider(new UsernamePasswordCredentialsProvider("debsap", "testpasword1"))
                .setDirectory(localPath)
                .call()) {

            Repository repository = git.getRepository();
            // create the folder
            File theDir = new File(repository.getDirectory().getParent(), "dir1");
            theDir.mkdir();

         // create the file
            File myfile = new File(theDir, "testfile2.txt");
            if(!myfile.createNewFile()) {
                throw new IOException("Could not create file " + myfile);
            }

            // Stage all files in the repo including new files
            git.add().addFilepattern(".").call();

            // and then commit the changes.
            git.commit().setMessage("Commit all changes including additions").call();

            try(PrintWriter writer = new PrintWriter(myfile)) {
                writer.append("Hello, world!");
            }
            // Stage all changed files, omitting new files, and commit with one command
            git.commit()
                    .setAll(true)
                    .setMessage("Commit changes to all files")
                    .call();
         // now open the created repository
            FileRepositoryBuilder builder = new FileRepositoryBuilder();
            try (Repository repository1 = builder.setGitDir(localPath)
                    .readEnvironment() // scan environment GIT_* variables
                    .findGitDir() // scan up the file system tree
                    .build()) {

                try (Git git1 = new Git(repository1)) {
                    git1.push().call();
                }
                System.out.println("Pushed from repository: " + repository1.getDirectory() + " to remote repository at " + REMOTE_URL);
            }
        }
    }
}

I am getting below error while trying to push to remote repo or github.

Cloning from https://github.com/debsap/testrepo.git to C:\Users\ramgood\AppData\Local\Temp\REStGitRepository8321744366017013430 Exception in thread "main" org.eclipse.jgit.api.errors.TransportException: origin: not found. at org.eclipse.jgit.api.PushCommand.call(PushCommand.java:183) at org.dstadler.jgit.unfinished.PushToRemoteRepository.main(PushToRemoteRepository.java:88) Caused by: org.eclipse.jgit.errors.NoRemoteRepositoryException: origin: not found. at org.eclipse.jgit.transport.TransportLocal$1.open(TransportLocal.java:131) at org.eclipse.jgit.transport.TransportBundleFile$1.open(TransportBundleFile.java:106) at org.eclipse.jgit.transport.Transport.open(Transport.java:565) at org.eclipse.jgit.transport.Transport.openAll(Transport.java:383) at org.eclipse.jgit.api.PushCommand.call(PushCommand.java:147) ... 1 more

Below is the config file from .git folder. Note that i cannot edit this manually and should be updated through java program only.

[core] symlinks = false repositoryformatversion = 0 filemode = false logallrefupdates = true [remote "origin"] url = https://github.com/debsap/testrepo.git fetch = +refs/heads/:refs/remotes/origin/ [branch "master"] remote = origin merge = refs/heads/master

Edit(Solution): Below is the working program.

public void pushToRemote(String xmlViewContent) throws IOException, InvalidRemoteException, TransportException, GitAPIException{
        // prepare a new folder for the cloned repository
        File localPath = File.createTempFile("GitRepository", "");
        if(!localPath.delete()) {
            throw new IOException("Could not delete temporary file " + localPath);
        }

        // then clone
        System.out.println("Cloning from " + REMOTE_URL + " to " + localPath);
        try (Git git = Git.cloneRepository()
                .setURI(REMOTE_URL)
                .setCredentialsProvider(new UsernamePasswordCredentialsProvider(username, password))
                .setDirectory(localPath)
                .call()) {

            Repository repository = git.getRepository();
            // create the folder
            File theDir = new File(repository.getDirectory().getParent(), "webapp");
                theDir.mkdir();

         // create the file
            File myfile = new File(theDir, "InputView.view.xml");
                myfile.createNewFile();

            // Stage all files in the repo including new files
            git.add().addFilepattern(".").call();

            // and then commit the changes.
            git.commit().setMessage("Commit all changes including additions").call();

            try(PrintWriter writer = new PrintWriter(myfile)) {
                writer.append( xmlViewContent );
            }
            // Stage all changed files, omitting new files, and commit with one command
            git.commit()
                    .setAll(true)
                    .setMessage("Commit changes to all files")
                    .call();
            git.add().addFilepattern("*").call();
            RevCommit result = git.commit().setMessage("initial commit").call();
            git.push()
                .setCredentialsProvider(new UsernamePasswordCredentialsProvider(username, password))
                .call();
            System.out.println("Pushed with commit: "+ result);

        }
    }

Thanks to VonC for his responses.

Answer 1

For manipulating files in a Git repo, you might be better off using JGit directly, not Egit-GitHub (though it is based on JGit).

That way, you can benefit from the examples in centic9/jgit-cookbook, including a jgit/porcelain/CommitAll.java example using addFilepattern:

// Stage all files in the repo including new files
git.add().addFilepattern(".").call();

That way, you can add any folder (content) you need.

I am running this as a service in a cloud instance where I dont know if I can rely on filerepo which is refererred in the cookbook example. If this works in cloud, i can consider that as a solution.

You will rely on the same file repo you are using right now.

The example is also adding a simple file but not a folder/directory.

Simply replace the file by a folder: it will work and add the folder content.

And I need to push changes to GitHub using a java program, not git commands. How can I do that?

Simply by adding a remote to the GitHub repo, and pushing.
Both are available in JGit (so in Java).