Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to free up disk space, when deleting files from Jupyter notebook

When I am removing files from Jupyter notebook environment, the disk space does not free up. I removed for about 40GB files and files disappeared from list, even from ls -a, however df -h shows that nothing happened. Also I killed all the processes using these files and even rebooted the system.

When I remove files using rm everything is fine. How can I free up space, or restore thos files to delete them using rm?

like image 792
BatyaGG Avatar asked Aug 20 '20 05:08

BatyaGG


People also ask

How do I clear the memory on my Jupyter notebook?

Jupyter Lesson 10: How to reset the Kernel (Clear all in memory objects and stop the code) If the notebook or your code is acting weird sometimes it is best to press the “RESET BUTTON”. Reseting the kernel clears all in memory objects and restarts your code from the very top.

How do I delete files from Jupyter notebook?

You can delete Jupyter Notebook files (. ipynb) from the dashboard by selecting the checkbox to left of the filename and then selecting the red trash can button that appears in the top left of the dashboard menu.

How much space does Jupyter Notebook take?

Memory and disk space required per user: 512MB RAM + 1GB of disk + . 5 CPU core. Server overhead: 2-4GB or 10% system overhead (whatever is larger), . 5 CPU cores, 10GB disk space.

Where does Jupyter delete files from a notebook?

Be aware that files deleted from within a Jupyter notebook (i.e. using menu functionality) are not deleted but moved to /home/username/.local/share/Trash/ (where "username" is the name of the user running the notebook, e.g. "ubuntu").

How to fix “deleting files doesn't free up space”?

1 Right-click on the Windows icon and select "Disk Management". 2 Right-click on the target drive or partition and click on "Format". 3 Select the file system and set the cluster size. 4 Click "OK" to format the drive. If this error still exists, you can consider wiping this disk to fix "deleting files doesn't free up space".

How do I free up space on my hard drive?

Thus, to free up space, you need to go to your computer's trash folder to completely delete those files. In Linux, you can do rm -rf $HOME/.local/share/Trash/files to empty trash.

How to recover disk space in Windows 10?

1. Press Windows + R to open Run window. 2. Type %temp% >Enter. 3. Select all the files that you see under this folder and hit Delete. This will remove all temporary files, helping recover storage space. Note: You might get a message that files are in use.


Video Answer


2 Answers

I also encountered the same issue, but later found out that files deleted in jupyter notebook are automatically moved to the trash, which is not permanently deleted instantly. This function was added very early ago: https://github.com/jupyter/notebook/pull/1968

Thus, to free up space, you need to go to your computer's trash folder to completely delete those files. In Linux, you can do rm -rf $HOME/.local/share/Trash/files to empty trash.

In Windows or MacOS, you just need to "Empty Trash" in the desktop folder

To restore those files, try to access them in your Trash folder, which is located in the .local folder in your home directory (in my system.).

This worked for me. I'm using jupyter lab with Amazon Linux 2 AMI.

Ref.

  1. recycle bin in linux: https://www.pcquest.com/recycle-bin-linux/#:~:text=Henceforth%20whenever%20you%20delete%20a,SYSTEM_ROOT%20of%20the%20Trash%20directory.
like image 119
Dwa Avatar answered Oct 19 '22 17:10

Dwa


I freed up the space and solved the issue when I was working with workbench(Google Cloud Vertex AI). Applicable to AI platform too. Open terminal from workbench File>New>Terminal

df -h -T /home/jupyter

Above code helps to know the free size before

Then this code below will delete the trash to free up the size

sudo rm -rf  /home/jupyter/.local/share/Trash/*

After deleting the trash after

like image 45
Partha Sen Avatar answered Oct 19 '22 19:10

Partha Sen