Transfer large file via SFTP in PHP

Question

I have a large file (200 MB upwards). I need to transfer it via PHP cron job. Using Phpseclib gives the following error:

Allowed memory size of 134217728 bytes exhausted (tried to allocate 4133 bytes) in /app/vendor/phpseclib/phpseclib/phpseclib/Net/SSH2.php

Is there a way I can do this with PHP cron job?

The code is simple one line where $localFile is an already existing CSV file

$sftp->put('/Import/coupons/coupons_import_test.csv', $localFile, NET_SFTP_LOCAL_FILE);

PS. This needs to be done after PHP generates that file in the /tmp folder so timing of the transfer script has to come into play.

[Edit] I do not intend on increasing the memory limit as the files later could be of higher sizes. A solution where I can transfer the file in parts (append mode) or use some shell script with PHP cron could be worthwhile

The file size on remote server is 111.4 MB while the actual file is much larger on local.

[Edit after the fix] The issue disappeared after changing to version 2.0.2 from version 1.0 I had to modify the code for put

$sftp->put('/Import/coupons/coupons_import.csv', $localFile, $sftp::SOURCE_LOCAL_FILE);

Answer 1

Phpseclib should do just fine transferring big files with no need to increase available memory.

I think you probably hit the old bug "SSH2: don't count data length against window size". You most likely may be using older version of the Phpseclib ( The older faulty version is bundled even with relatively new software like for example Magento 1.9.* )

Check your version if is not the latest redownload it from https://github.com/phpseclib/phpseclib

Answer 2

In lieu of seeing the code that you're using my guess is that you're trying to load the 200mb file as a string and you're trying to upload it as a string. eg. $sftp->put('filename.remote', file_get_contents('filename.local'));. If so then try this instead:

$sftp->put('filename.remote', 'filename.local', NET_SFTP_LOCAL_FILE);