Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Error: read ECONNRESET , while running the python child process from nodejs

I am getting the error while running python child process on EC2 Ubuntu from node js. The spawned child process is never invoked by node.js, however things are working perfectly fine on local. We initially were getting an EPIPE error, until we used 'sudo apt-get install libfontconfig' which then produced this error below:

events.js:160
      throw er; // Unhandled 'error' event
      ^ //error Error: read ECONNRESET
    at exports._errnoException (util.js:1020:11)
    at Pipe.onread (net.js:568:26)



        //child process             
        urlCrawlJob(hostname, pageCounter+1, accessToken); //recursive calling of the function


        var process = SPAWN('python', [PATH.join(__dirname,"../pyScripts/crawler.py")]),

        data = body.customers;
        dataString = '';

        console.log(`Spawned child pid: ${process.pid}`);
        process.stdout.on('error', function (err) {
            console.log('stdout error: ', err);
            console.log(err.code);
        });

        process.stdout.on('data', function(data){
            dataString+=data
            console.log(dataString);
        });
        process.stdout.on('end', function(){

            console.log("ending child process -----> call url");

        });
        process.stdin.write(JSON.stringify({"data":data,"hostname":hostname}));
        process.stdin.end();                

    //python script

## process_init.py

#crete seperate function, impletment oop concepet

import sys, json, pymongo, os


#defaukt address, email, phnNo, total spe

print "==> in crawler python"

#print sys.args[0];

def main():
    data = json.load(sys.stdin)
    hostname = data['hostname'];
    customerData = data['data'];

    print hostname


    collection = dbConnection(hostname)


    #isFile = open("/crawlerResult/"+hostname+".txt","w+")


    pwd = os.getcwd()

    print pwd
    file = open(pwd+"/crawlerResult/"+hostname+".txt", "a++")

    iteratingData(customerData, file, collection)


    print "=====\n\n"
    client.close();                             #close the db connection
    sys.stdout.flush();  

def dbConnection(hostname):

    #opening connection with db
    client = pymongo.MongoClient('127.0.0.1', 27017) ;                          # TODO: if connection already open do not open neew one
    #   print client
    db = client["customerLTV"];
    print db
    collection = db[hostname];
    return collection


def iteratingData(customerData, file, collection):

    count = 0
    data = {}
    for i in customerData:
        #print i
        count += 1
        #print len(i["addresses"]);

        try:


            strdata = {}
            strdata[count] = data   
            strdata = json.dumps(strdata)
            file.write(strdata + "\n")                  


            data.pop('_id', None)
            collection.insert(data)         
            data = {}
        except Exception, e:
            print str(e)

    print count;




#start process
if __name__ == '__main__':
    main()
like image 523
Kushan Avatar asked Aug 08 '26 03:08

Kushan


1 Answers

It looks like your modules are not installed correctly on the Ubuntu server. Did you install or update all the Ubuntu EC2 modules? Especially if the Python child processes are not running.

like image 123
Luca Lavorato Avatar answered Aug 09 '26 15:08

Luca Lavorato



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!