Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

nodejs how to get JSON instead of Buffer from aws s3 bucket

I have a .json file stored in a S3 Bucket, now i Want to download the json File with nodejs. I wrote following Code:

const bucket = "s3bucketname";

const AWS = require('aws-sdk');
const S3= new AWS.S3();
exports.handler = async (event, context, callback) => {
    var transcript = await download();
    console.log(transcript);
}

async function download(){
  try {
    // Converted it to async/await syntax just to simplify.
    const data = await S3.getObject(
    {   Bucket: bucket, 
        Key: "Test.json",
        //ResponseContentType: 'application/json'
    }).promise();

    console.log(data);
    return {
      statusCode: 200,
      body: JSON.stringify(data)
    }
  }
  catch (err) {
    return {
      statusCode: err.statusCode || 400,
      body: err.message || JSON.stringify(err.message)
    }
  }
}

My Response is like this: AcceptRanges: 'bytes', LastModified: 2020-02-07T08:04:25.000Z, ContentLength: 12723, ETag: '"ea7de645f93c45b3jkj4e7ffjdsf"', ContentType: 'application/octet-stream', Metadata: {}, Body: Buffer 7b 0d ...

In the body i get a Buffer from my JSON, if I convert it via tools like: https://onlineutf8tools.com/convert-bytes-to-utf8

I get my JSON string like i want it. How can i do that in Javascript/nodejs ? I don't need the Buffer, I need the JSON in String. I tried different ways, but it didnt work yet.

like image 307
aw123 Avatar asked Feb 11 '20 11:02

aw123


People also ask

How do I pull data from AWS S3 bucket?

In the Amazon S3 console, choose your S3 bucket, choose the file that you want to open or download, choose Actions, and then choose Open or Download. If you are downloading an object, specify where you want to save it. The procedure for saving the object depends on the browser and operating system that you are using.

Can you store JSON in S3?

Amazon S3 Select works on objects stored in CSV, JSON, or Apache Parquet format. It also works with objects that are compressed with GZIP or BZIP2 (for CSV and JSON objects only), and server-side encrypted objects.

What is get object in S3?

PDF. Retrieves objects from Amazon S3. To use GET , you must have READ access to the object. If you grant READ access to the anonymous user, you can return the object without using an authorization header.

What is JSON document in AWS?

JavaScript Object Notation, more commonly known by the acronym JSON, is an open data interchange format that is both human and machine-readable. Despite the name JavaScript Object Notation, JSON is independent of any programming language and is a common API output in a wide variety of applications.


1 Answers

I don't have the reputation to add comment for @Ashish Modi's answer. The data.Body.toString('utf-8') will convert it to plain string not to json object.
If you want to have the json object you can convert the string to json object by JSON.parse(obj.Body.toString('utf-8'))

like image 87
user1497277 Avatar answered Sep 17 '22 12:09

user1497277