tensorflow random forest regression

Question

I would like to implement a simple random forest regression to predict a value. The inputs are some samples with several features, and the label is a value. However, I cannot find a simple example about the random forest regression problem. Thus, I saw the document of tensorflow and I found that:

An estimator that can train and evaluate a random forest. Example:

  python
  params = tf.contrib.tensor_forest.python.tensor_forest.ForestHParams(
      num_classes=2, num_features=40, num_trees=10, max_nodes=1000)
  # Estimator using the default graph builder.
  estimator = TensorForestEstimator(params, model_dir=model_dir)
  # Or estimator using TrainingLossForest as the graph builder.
  estimator = TensorForestEstimator(
      params, graph_builder_class=tensor_forest.TrainingLossForest,
      model_dir=model_dir)
  # Input builders
  def input_fn_train: # returns x, y
    ...
  def input_fn_eval: # returns x, y
    ...
  estimator.fit(input_fn=input_fn_train)
  estimator.evaluate(input_fn=input_fn_eval)
  # Predict returns an iterable of dicts.
  results = list(estimator.predict(x=x))
  prob0 = results[0][eval_metrics.INFERENCE_PROB_NAME]
  prediction0 = results[0][eval_metrics.INFERENCE_PRED_NAME]

However, when I follow the example, I got the error on the line,prob0 = results[0][eval_metrics.INFERENCE_PROB_NAME], the error shows that:

Example conversion:
est = Estimator(...) -> est = SKCompat(Estimator(...))
Traceback (most recent call last):
  File "RF_2.py", line 312, in <module>
    main()
  File "RF_2.py", line 298, in main
    train_eval(x_train, y_train, x_validation, y_validation, x_test, y_test, num_tree)
  File "RF_2.py", line 221, in train_eval
    prob0 = results[0][eval_metrics.INFERENCE_PROB_NAME]
KeyError: 'probabilities'

I think the error occurs on INFERENCE_PROB_NAME, and I saw the document. However, I still do not know what the word is to replace INFERENCE_PROB_NAME.

I have tried get_metric('accuracy') to replace INFERENCE_PROB_NAME, it return the error: KeyError: <function _accuracy at 0x11a06eaa0>.

I also tried get_prediction_key('accuracy') to replace INFERENCE_PROB_NAME, it return the error: KeyError: 'classes'.

If you know the possible answer, please tell me. Thank you in advance.

Answer 1

I think you are unintentionally doing a classification problem by giving a wrong num_classes=2 and not changing the default value of regression=False. See the Parameters section here. Just as a quick test, set num_classes=0 and regression=True, and re-run your code.

Answer 2

num_classes=0 is wrong in tensorflow 1.3.0.

From the link of Mehdi Rezaie, num_classes is the number of dimensions in the output of a regression problem.

You have to use num_classes=1 or bigger value for num_classes. Or You'll get the error like ValueError: Invalid logits_dimension 0.