Reading java file with escape characters for newline

Question

I have a Unicode file that needs to be exported to database(Vertica). The column delimiter is CTRL+B, record delimiter is newline(\n). Whenever there is a newline within a column value, CTRL+A is used as escape character.

When I use BufferedReader.readLine() to read this file, the records with ID's 2 and 4, are read as two records. Whereas I want to read them as a single whole record as given in output.

Here is the example input file. | stands for CTRL+B and ^ stands for CTRL+A.

Input
ID|Name|Job Desc
----------------
1|xxxx|SO Job
2|YYYY|SO Careers^
Job
3|RRRRR|SO
4|ZZZZ^
 ZZ|SO Job
5|AAAA|YU

Output:
ID|Name|Job Desc
----------------
1|xxxx|SO Job
2|YYYY|SO Careers Job
3|RRRRR|SO
4|ZZZZ ZZ|SO Job
5|AAAA|YU

The file is huge, so I cant use StringEscapeUtils. Any suggestions on this?

Answer 1

You can use a Scanner with a custom delimeter. The delimeter I use is set to match \n but not \u0001\n (where \u0001 represents CTRL+A):

try {
    PrintWriter writer = new PrintWriter("dboutput.txt");
    Scanner sc = new Scanner(new File("dbinput.txt"));
    sc.useDelimiter(Pattern.compile("^(?!.*(\\u0001\\n)).*\\n$"));
    while (sc.hasNext()) {
        writer.println(sc.next());
    }
    scanner.close();
    writer.close();
} catch (FileNotFoundException e) {
   e.printStackTrace();
}