I have a problem in running Hadoop pipes code on Mac. This is my c++ code.
#include <algorithm>
#include <limits>
#include <stdint.h>
#include <string>
#include "Pipes.hh"
#include "TemplateFactory.hh"
#include "StringUtils.hh"
using namespace std;
class MaxTemperatureMapper : public HadoopPipes::Mapper {
public:
MaxTemperatureMapper(HadoopPipes::TaskContext& context) {
}
void map(HadoopPipes::MapContext& context) {
std::string line = context.getInputValue();
std::string year = line.substr(15, 4);
std::string airTemperature = line.substr(87, 5);
std::string q = line.substr(92, 1);
if (airTemperature != "+9999" &&
(q == "0" || q == "1" || q == "4" || q == "5" || q == "9")) {
context.emit(year, airTemperature);
}
}
};
class MapTemperatureReducer : public HadoopPipes::Reducer {
public:
MapTemperatureReducer(HadoopPipes::TaskContext& context) {
}
void reduce(HadoopPipes::ReduceContext& context) {
int maxValue = INT_MIN;
while (context.nextValue()) {
maxValue = std::max(maxValue, HadoopUtils::toInt(context.getInputValue()));
}
context.emit(context.getInputKey(), HadoopUtils::toString(maxValue));
}
};
int main(int argc, char * argv[]) {
int i=HadoopPipes::runTask(HadoopPipes::TemplateFactory<WordCountMap, WordCountReduce>()); // 运行任务
return 0;
}
My makefile is:
wordcount :wordcount.cpp
g++ -Wall -I/Users/macbookpro/Documents/hadoop-2.7.5/include -L/Users/macbookpro/Documents/hadoop-2.7.5/lib/native -lhadooppipes -lhadooputils -lpthread -lcrypto -lssl -g -O2 -o $@
When I tried to compile the source file, I got an error saying that
Undefined symbols for architecture x86_64: "_main", referenced from: implicit entry/start for main executable ld: symbol(s) not found for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) make: *** [wordcount] Error 1
I don't know what this means because I already have a main function.
could you solve this problem?
Thanks!
You didn't pass the source file to the recipe, the easiest way to avoid this here is to just rely on the built-in make rule for one-shotting programs, all you need is:
CPPFLAGS := -I/Users/macbookpro/Documents/hadoop-2.7.5/include -pthread
CXXFLAGS := -Wall -g -O2
LDFLAGS := -L/Users/macbookpro/Documents/hadoop-2.7.5/lib/native -pthread
LDLIBS := -lhadooppipes -lhadooputils -lcrypto -lssl
wordcount:
The make implicit rule for %: %.cpp will take care of the rest. Note that you're using pthread incorrectly, you need to pass the pthread option to the preprocessor and linker, not the lib.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With