When I am removing files from Jupyter notebook environment, the disk space does not free up. I removed for about 40GB files and files disappeared from list, even from ls -a
, however df -h
shows that nothing happened. Also I killed all the processes using these files and even rebooted the system.
When I remove files using rm
everything is fine. How can I free up space, or restore thos files to delete them using rm
?
Jupyter Lesson 10: How to reset the Kernel (Clear all in memory objects and stop the code) If the notebook or your code is acting weird sometimes it is best to press the “RESET BUTTON”. Reseting the kernel clears all in memory objects and restarts your code from the very top.
You can delete Jupyter Notebook files (. ipynb) from the dashboard by selecting the checkbox to left of the filename and then selecting the red trash can button that appears in the top left of the dashboard menu.
Memory and disk space required per user: 512MB RAM + 1GB of disk + . 5 CPU core. Server overhead: 2-4GB or 10% system overhead (whatever is larger), . 5 CPU cores, 10GB disk space.
Be aware that files deleted from within a Jupyter notebook (i.e. using menu functionality) are not deleted but moved to /home/username/.local/share/Trash/ (where "username" is the name of the user running the notebook, e.g. "ubuntu").
1 Right-click on the Windows icon and select "Disk Management". 2 Right-click on the target drive or partition and click on "Format". 3 Select the file system and set the cluster size. 4 Click "OK" to format the drive. If this error still exists, you can consider wiping this disk to fix "deleting files doesn't free up space".
Thus, to free up space, you need to go to your computer's trash folder to completely delete those files. In Linux, you can do rm -rf $HOME/.local/share/Trash/files to empty trash.
1. Press Windows + R to open Run window. 2. Type %temp% >Enter. 3. Select all the files that you see under this folder and hit Delete. This will remove all temporary files, helping recover storage space. Note: You might get a message that files are in use.
I also encountered the same issue, but later found out that files deleted in jupyter notebook are automatically moved to the trash, which is not permanently deleted instantly. This function was added very early ago: https://github.com/jupyter/notebook/pull/1968
Thus, to free up space, you need to go to your computer's trash folder to completely delete those files. In Linux, you can do rm -rf $HOME/.local/share/Trash/files
to empty trash.
In Windows or MacOS, you just need to "Empty Trash" in the desktop folder
To restore those files, try to access them in your Trash folder, which is located in the .local folder in your home directory (in my system.).
This worked for me. I'm using jupyter lab with Amazon Linux 2 AMI.
Ref.
I freed up the space and solved the issue when I was working with workbench(Google Cloud Vertex AI). Applicable to AI platform too. Open terminal from workbench File>New>Terminal
df -h -T /home/jupyter
Above code helps to know the free size
Then this code below will delete the trash to free up the size
sudo rm -rf /home/jupyter/.local/share/Trash/*
After deleting the trash
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With