Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What does tf.gfile do in TensorFlow?

I've seen people using several functions from tf.gfile such as tf.gfile.GFile or tf.gfile.Exists. I have the idea that tf.gfile deals with files. However, I haven't been able to find the official documentation to see what else it offers.

It'd be great if you could help me with it.

like image 868
Chip Huyen Avatar asked Feb 15 '17 18:02

Chip Huyen


People also ask

What is GFile Tensorflow?

gfile module are: To provide an API that is close to Python's file I/O objects, and. To provide an implementation based on TensorFlow's C++ FileSystem API.

What is a GFile?

GFile is a high level abstraction for manipulating files on a virtual file system. GFiles are lightweight, immutable objects that do no I/O upon creation. It is necessary to understand that GFile objects do not represent files, merely an identifier for a file.


1 Answers

For anyone landing here, the following answer was provided (by a googler) on: Why use tensorflow gfile? (for file I/O)

The main roles of the tf.gfile module are:

  1. To provide an API that is close to Python's file objects, and

  2. To provide an implementation based on TensorFlow's C++ FileSystem API.

The C++ FileSystem API supports multiple file system implementations, including local files, Google Cloud Storage (using a gs:// prefix), and HDFS (using an hdfs:// prefix). TensorFlow exports these as tf.gfile, so that you can use these implementations for saving and loading checkpoints, writing TensorBoard logs, and accessing training data (among other uses). However, if all of your files are local, you can use the regular Python file API without any problem.

like image 81
Yuval Atzmon Avatar answered Oct 05 '22 13:10

Yuval Atzmon