Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to "reset" tensorboard data after killing tensorflow instance

I'm testing different hyperparameters for a cnn model I built, but I'm having a small annoyance when viewing the summaries in Tensorboard. The problem seems to be that the data is just "added" in consecutive runs, so the functions result in a weird superposition unless I see the information as "relative" instead of "by step". See here:

X Type: Step

X Type: Relative

I've tried killing tensorboard's process and erasing the log files, but it seems it is not enough.

So the question is, how do I reset this information?

Thanks!!

like image 805
mathetes Avatar asked Dec 24 '15 15:12

mathetes


People also ask

How do you refresh a TensorBoard?

The reload interval can be configured using the --reload_interval flag of the TensorBoard process, but this option is currently only available in master and as of version 0.8 has not been released.

How do I delete runs in TensorBoard?

Deleting your TensorBoard. To remove an experiment you have uploaded, use the delete command and specify the appropriate experiment_id .

How does TensorBoard save data?

Just check the "Data download links" option on the upper-left in TensorBoard, and then click on the "CSV" button that will appear under your scalar summary.

What is Logdir in TensorBoard?

tensorboard --logdir=summaries. --logdir is the directory you will create data to visualize. Files that TensorBoard saves data into are called event files. Type of data saved into the event files is called summary data. Optionally you can use --port=<port_you_like> to change the port TensorBoard runs on.


1 Answers

Note: The solution you've posted (erase TensorBoard's log files and kill the process) will work, but it isn't preferred, because it destroys historical information about your training.

Instead, you can have each new training job write to a new subdirectory (of your top-level log directory). Then, TensorBoard will consider each job a new "run" and will create a nice comparison view so you can see how the training differed between iterations of your model.

In the following an example from https://www.tensorflow.org/tensorboard/get_started:

model = create_model() ... model.compile(...)  log_dir = "logs/fit/" + datetime.datetime.now().strftime("%Y%m%d-%H%M%S") tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)  model.fit(..., callbacks=[tensorboard_callback]) 
like image 176
dandelion Avatar answered Oct 25 '22 09:10

dandelion